From 5c05455948236c756f1212e54b19d8c047675bbf Mon Sep 17 00:00:00 2001 From: araf-statsig Date: Mon, 16 Feb 2026 23:42:28 -0800 Subject: [PATCH 1/6] eval scoring improvements --- messages/evaluationScoring.mdx | 109 ++++++++++++++++++++++++++++----- 1 file changed, 92 insertions(+), 17 deletions(-) diff --git a/messages/evaluationScoring.mdx b/messages/evaluationScoring.mdx index 4a773ed31..901ad4dd2 100644 --- a/messages/evaluationScoring.mdx +++ b/messages/evaluationScoring.mdx @@ -2,30 +2,76 @@ title: Evaluation Cost Breakdown --- -Feature gate evaluation speeds are determined through the overall complexity of all rules included in the gate. Each condition, operator, and quantity of values are scored through a point system and used to classify individual rules and gates as normal, slow, or very slow. These factors are weighted with the worst case scenario in mind. The tables below can be used to understand which component of your rule is making your evaluation slow. +Feature gate evaluation speeds are determined by the overall complexity of all rules in a gate. Each condition, operator, and value count contributes to a score that classifies rules and gates as Normal, Slow, or Very Slow. Use this page to quickly identify the likely cause of a slow gate and the fastest way to fix it. -## Condition +## How to Interpret "Slow" and "Very Slow" + +These labels are relative tiers, not fixed millisecond thresholds. Actual runtime depends on SDK language, device/server, and whether the SDK needs to infer attributes (for example, user-agent parsing or IP geolocation). To translate a label into time on **your** stack, measure it directly on the same server or device class you care about. + + + +1. Pick a representative server type or device and a production-like `StatsigUser`. +1. Warm up the SDK (initialize and run a few checks). +1. Measure a baseline gate (for example, a gate with no rules) for N evaluations and record p50/p95. +1. Measure your slow gate for the same N evaluations and record p50/p95. +1. The delta between the two is the true impact for your environment. + + + +## Quick Diagnosis + +| What you see | Likely cause | Fastest fix | +| --- | --- | --- | +| Gate targets OS/Browser/Version and is slow | User-agent parsing at evaluation time | Pass `os_name`, `os_version`, `browser_name`, `browser_version` (server) or `systemName`, `systemVersion`, `browserName`, `browserVersion` (client) directly on `StatsigUser` | +| Gate targets Country and is slow | IP geolocation lookup | Pass `country` directly on `StatsigUser` | +| Gate uses large ID lists (overrides or segment rules) | Large list lookup | Trim stale IDs, split lists, or move to rule-based inline segments or a custom attribute | +| Gate uses Regex or Version String operators | Expensive per-check parsing | Replace with exact matches or precomputed attributes | +| Gate has many nested Passes Gate/Segment rules | Costs compound across nested evaluations | Reduce nesting or push logic into a single gate/segment | +| Conditions have many values (50+) | Large value scans | Prune values, split cohorts, or move to a smaller segment | + +## Cost Drivers and Fixes + +### Conditions These costs are applied based on the **condition type** used in a gate. Only conditions with a significant impact are listed. If a condition is not listed, it does not significantly impact SDK evaluation speeds. -| Condition | Impact | Reason | -| --------------- | -------- | ----------------------------------------------- | -| BROWSER | High | Requires user agent parsing | -| BROWSER VERSION | High | Requires user agent parsing | -| OS | High | Requires user agent parsing | -| OS VERSION | High | Requires user agent parsing | -| COUNTRY | Low | Requires IP-based geolocation lookup | -| PASSES SEGMENT | Variable | complexity dependent on cost of nested segments | -| FAILS SEGMENT | Variable | complexity dependent on cost of nested segments | -| PASSES GATE | Variable | complexity dependent on cost of nested gates | -| FAILS GATE | Variable | complexity dependent on cost of nested gates | +| Condition | Impact | Reason | +| --------------- | -------- | ---------------------------------------------------------------------------- | +| BROWSER | Variable | Requires user agent parsing unless Browser is explicitly provided | +| BROWSER VERSION | Variable | Requires user agent parsing unless Browser Version is explicitly provided | +| OS | Variable | Requires user agent parsing unless Operating System is explicitly provided | +| OS VERSION | Variable | Requires user agent parsing unless OS Version is explicitly provided | +| COUNTRY | Low | Requires IP-based geolocation lookup when `country` is not explicitly set | +| PASSES SEGMENT | Variable | complexity dependent on cost of nested segments | +| FAILS SEGMENT | Variable | complexity dependent on cost of nested segments | +| PASSES GATE | Variable | complexity dependent on cost of nested gates | +| FAILS GATE | Variable | complexity dependent on cost of nested gates | + +Tips: +- If you target OS/Browser or their versions, pass explicit values on `StatsigUser` to avoid user-agent parsing. Server SDKs support `os_name`, `os_version`, `browser_name`, and `browser_version`; client SDKs use `systemName`, `systemVersion`, `browserName`, and `browserVersion`. +- If you target Country, pass `country` directly to avoid IP-based lookup. For conditions that depend on other gates or segments, their evaluation cost is directly related to the entity they rely on. If a gate has a **Passes Gate** condition, the evaluation cost of the gate it is referencing will be propagated up to the base **Passes Gate** rule. -## Operator + +### ID List Size (Overrides and Segment Conditions) + +ID lists have a direct impact on evaluation performance. This applies both to **ID list overrides** and **segment conditions** that reference ID lists. + +| ID List Size | Impact | +| ----------- | ------ | +| 1,000+ IDs | High | +| 10,000+ IDs | Very High | + +Tips: +- Trim stale IDs and scope lists to active cohorts. +- Split large lists into smaller, purpose-built cohorts. +- Prefer rule-based inline segments or custom attributes where possible. + +### Operator These costs are applied based on the **operator** used in a condition. If an operator is not listed, it does not significantly impact SDK evaluation speeds. @@ -37,13 +83,42 @@ These costs are applied based on the **operator** used in a condition. If an ope | Version String | Medium | Requires semantic version comparison | | ARRAY CONTAINS ANY / NONE / ALL | Low | Requires array scan | -## Value Count +Tips: +- Match Regex String: Replace with `ANY OF` or precompute a boolean/enum attribute (for example, `is_enterprise_email` or `email_domain`). +- Version String: Precompute a normalized numeric value or `major`/`minor` fields and compare those instead. +- ARRAY CONTAINS: Keep arrays short and consider flattening frequent checks into a single boolean attribute (for example, `has_beta_access: true`). + +### Value Count These costs are applied based on the **number of values** provided to a condition. | Entity | Impact | | ------------ | ------ | -| 10–19 values | low | -| 20–29 values | low | +| 10–19 values | Low | +| 20–29 values | Low | | 30–49 values | Medium | | 50+ values | High | + +Tips: +- If you are in the 50+ range, prune stale values or split cohorts. +- Consider moving large lists into a smaller inline segment or a custom attribute computed upstream. + +## Examples + +### Example 1: OS/Browser Parsing + +Issue: Gate targets OS Version and Browser Version and is Slow on server SDKs. + +Fix: Pass explicit OS/Browser fields on `StatsigUser` so the SDK does not parse the user agent. + +### Example 2: Large ID List Override + +Issue: Gate uses an ID list override with 12,000 IDs and is Very Slow. + +Fix: Split the list into smaller cohorts or replace it with a rule-based inline segment (for example, a custom attribute like `has_paid_subscription: true`). + +### Example 3: Regex Targeting + +Issue: Gate uses Match Regex String on email domain and is Slow. + +Fix: Precompute `email_domain` (or `is_enterprise_email`) and use `ANY OF` instead. From 266213d5a15e7003d7372f66848b34beac255aea Mon Sep 17 00:00:00 2001 From: araf-statsig Date: Tue, 17 Feb 2026 15:06:10 -0800 Subject: [PATCH 2/6] more edits --- messages/evaluationScoring.mdx | 94 ++++++++++++---------------------- 1 file changed, 32 insertions(+), 62 deletions(-) diff --git a/messages/evaluationScoring.mdx b/messages/evaluationScoring.mdx index 901ad4dd2..477431b9e 100644 --- a/messages/evaluationScoring.mdx +++ b/messages/evaluationScoring.mdx @@ -2,32 +2,22 @@ title: Evaluation Cost Breakdown --- -Feature gate evaluation speeds are determined by the overall complexity of all rules in a gate. Each condition, operator, and value count contributes to a score that classifies rules and gates as Normal, Slow, or Very Slow. Use this page to quickly identify the likely cause of a slow gate and the fastest way to fix it. +Feature gate evaluation speeds are determined by the overall complexity of all rules in a gate. Each **condition**, **operator**, and **value count** contributes to a score that classifies rules and gates as Normal, Slow, or Very Slow. Use this page to quickly identify the likely cause of a slow gate and the fastest way to fix it. ## How to Interpret "Slow" and "Very Slow" -These labels are relative tiers, not fixed millisecond thresholds. Actual runtime depends on SDK language, device/server, and whether the SDK needs to infer attributes (for example, user-agent parsing or IP geolocation). To translate a label into time on **your** stack, measure it directly on the same server or device class you care about. - - - -1. Pick a representative server type or device and a production-like `StatsigUser`. -1. Warm up the SDK (initialize and run a few checks). -1. Measure a baseline gate (for example, a gate with no rules) for N evaluations and record p50/p95. -1. Measure your slow gate for the same N evaluations and record p50/p95. -1. The delta between the two is the true impact for your environment. - - +These labels are relative tiers, not fixed millisecond thresholds. Actual runtime depends on device/server, network, and whether the SDK is using the worst case scenario. (ex. even if country is included in the user object we assume the worst case scenario and that it is being ip geolocated). These labels are intended to be a warning. If you are using the best possible practice for your context, you can ignore them. ## Quick Diagnosis -| What you see | Likely cause | Fastest fix | -| --- | --- | --- | -| Gate targets OS/Browser/Version and is slow | User-agent parsing at evaluation time | Pass `os_name`, `os_version`, `browser_name`, `browser_version` (server) or `systemName`, `systemVersion`, `browserName`, `browserVersion` (client) directly on `StatsigUser` | -| Gate targets Country and is slow | IP geolocation lookup | Pass `country` directly on `StatsigUser` | -| Gate uses large ID lists (overrides or segment rules) | Large list lookup | Trim stale IDs, split lists, or move to rule-based inline segments or a custom attribute | -| Gate uses Regex or Version String operators | Expensive per-check parsing | Replace with exact matches or precomputed attributes | -| Gate has many nested Passes Gate/Segment rules | Costs compound across nested evaluations | Reduce nesting or push logic into a single gate/segment | -| Conditions have many values (50+) | Large value scans | Prune values, split cohorts, or move to a smaller segment | +| What you see | Likely cause | Fastest fix | +| ---------------------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Gate targets OS/Browser/Version and is slow | User-agent parsing at evaluation time | Pass `os_name`, `os_version`, `browser_name`, `browser_version` (server) or `systemName`, `systemVersion`, `browserName`, `browserVersion` (client) directly on `StatsigUser` | +| Gate targets Country and is slow | IP geolocation lookup | Pass `country` directly on `StatsigUser` | +| Gate uses large ID lists (overrides or segment rules) | Large list lookup | Trim stale IDs, split lists, or move to rule-based inline segments or a custom attribute | +| Gate uses Regex or Version String operators | Expensive per-check parsing | Replace with exact matches or precomputed attributes | +| Gate has many nested Passes Gate/Segment rules | Costs compound across nested evaluations | Reduce nesting or push logic into a single gate/segment | +| Conditions have many values (50+) | Large value scans | Prune values, split cohorts, or move to a smaller segment | ## Cost Drivers and Fixes @@ -35,17 +25,18 @@ These labels are relative tiers, not fixed millisecond thresholds. Actual runtim These costs are applied based on the **condition type** used in a gate. Only conditions with a significant impact are listed. If a condition is not listed, it does not significantly impact SDK evaluation speeds. -| Condition | Impact | Reason | -| --------------- | -------- | ---------------------------------------------------------------------------- | -| BROWSER | Variable | Requires user agent parsing unless Browser is explicitly provided | -| BROWSER VERSION | Variable | Requires user agent parsing unless Browser Version is explicitly provided | -| OS | Variable | Requires user agent parsing unless Operating System is explicitly provided | -| OS VERSION | Variable | Requires user agent parsing unless OS Version is explicitly provided | -| COUNTRY | Low | Requires IP-based geolocation lookup when `country` is not explicitly set | -| PASSES SEGMENT | Variable | complexity dependent on cost of nested segments | -| FAILS SEGMENT | Variable | complexity dependent on cost of nested segments | -| PASSES GATE | Variable | complexity dependent on cost of nested gates | -| FAILS GATE | Variable | complexity dependent on cost of nested gates | +| Condition | Impact | worst case | Reason | +| | (assuming worst case) | | | +| --------------- | --------------------- | ---------------------------------- | ---------------------------------------------------------------------------- | +| BROWSER | medium | user agent | Requires user agent parsing unless Browser is explicitly provided | +| BROWSER VERSION | medium | user agent | Requires user agent parsing unless Browser Version is explicitly provided | +| OS | medium | user agent | Requires user agent parsing unless Operating System is explicitly provided | +| OS VERSION | medium | user agent | Requires user agent parsing unless OS Version is explicitly provided | +| COUNTRY | Low | IP lookup | Requires IP-based geolocation lookup when `country` is not explicitly set | +| PASSES SEGMENT | high | deeply nested / large segment | complexity dependent on cost of nested segments | +| FAILS SEGMENT | high | deeply nested / large segment | complexity dependent on cost of nested segments | +| PASSES GATE | high | deeply nested with slow gates | complexity dependent on cost of nested gates | +| FAILS GATE | high | deeply nested with slow gates | complexity dependent on cost of nested gates | Tips: - If you target OS/Browser or their versions, pass explicit values on `StatsigUser` to avoid user-agent parsing. Server SDKs support `os_name`, `os_version`, `browser_name`, and `browser_version`; client SDKs use `systemName`, `systemVersion`, `browserName`, and `browserVersion`. @@ -61,15 +52,14 @@ For conditions that depend on other gates or segments, their evaluation cost is ID lists have a direct impact on evaluation performance. This applies both to **ID list overrides** and **segment conditions** that reference ID lists. -| ID List Size | Impact | -| ----------- | ------ | -| 1,000+ IDs | High | -| 10,000+ IDs | Very High | +| ID List Size | Impact | +| ------------ | --------- | +| 1,000+ IDs | High | +| 10,000+ IDs | Very High | Tips: - Trim stale IDs and scope lists to active cohorts. - Split large lists into smaller, purpose-built cohorts. -- Prefer rule-based inline segments or custom attributes where possible. ### Operator @@ -77,16 +67,16 @@ These costs are applied based on the **operator** used in a condition. If an ope | Operator | Impact | Reason | | ------------------------------- | ------ | --------------------------------------- | -| ANY OF (Case Sensitive) | Low | Requires explicit equality comparison | -| NONE OF (Case Sensitive) | Low | Requires explicit inequality comparison | +| Any of (Case Sensitive) | Low | Requires explicit equality comparison | +| None of (Case Sensitive) | Low | Requires explicit inequality comparison | | Match Regex String | Medium | Requires regex matching | | Version String | Medium | Requires semantic version comparison | -| ARRAY CONTAINS ANY / NONE / ALL | Low | Requires array scan | +| Array Contains Any / None / All | Low | Requires array scan | Tips: -- Match Regex String: Replace with `ANY OF` or precompute a boolean/enum attribute (for example, `is_enterprise_email` or `email_domain`). -- Version String: Precompute a normalized numeric value or `major`/`minor` fields and compare those instead. -- ARRAY CONTAINS: Keep arrays short and consider flattening frequent checks into a single boolean attribute (for example, `has_beta_access: true`). +- **Match Regex String**: Replace with `ANY OF` or precompute a boolean/enum attribute. +- **Version String**: Precompute a normalized numeric value or `major`/`minor` fields and compare those instead. +- **Array Contains**: Keep arrays short and consider flattening frequent checks into a single boolean attribute. ### Value Count @@ -102,23 +92,3 @@ These costs are applied based on the **number of values** provided to a conditio Tips: - If you are in the 50+ range, prune stale values or split cohorts. - Consider moving large lists into a smaller inline segment or a custom attribute computed upstream. - -## Examples - -### Example 1: OS/Browser Parsing - -Issue: Gate targets OS Version and Browser Version and is Slow on server SDKs. - -Fix: Pass explicit OS/Browser fields on `StatsigUser` so the SDK does not parse the user agent. - -### Example 2: Large ID List Override - -Issue: Gate uses an ID list override with 12,000 IDs and is Very Slow. - -Fix: Split the list into smaller cohorts or replace it with a rule-based inline segment (for example, a custom attribute like `has_paid_subscription: true`). - -### Example 3: Regex Targeting - -Issue: Gate uses Match Regex String on email domain and is Slow. - -Fix: Precompute `email_domain` (or `is_enterprise_email`) and use `ANY OF` instead. From e328d5dc88a1f75082e832d4e4f649b2ba5f55bd Mon Sep 17 00:00:00 2001 From: araf-statsig Date: Wed, 18 Feb 2026 14:55:10 -0800 Subject: [PATCH 3/6] polish --- messages/evaluationScoring.mdx | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/messages/evaluationScoring.mdx b/messages/evaluationScoring.mdx index 477431b9e..90d533901 100644 --- a/messages/evaluationScoring.mdx +++ b/messages/evaluationScoring.mdx @@ -14,7 +14,7 @@ These labels are relative tiers, not fixed millisecond thresholds. Actual runtim | ---------------------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | Gate targets OS/Browser/Version and is slow | User-agent parsing at evaluation time | Pass `os_name`, `os_version`, `browser_name`, `browser_version` (server) or `systemName`, `systemVersion`, `browserName`, `browserVersion` (client) directly on `StatsigUser` | | Gate targets Country and is slow | IP geolocation lookup | Pass `country` directly on `StatsigUser` | -| Gate uses large ID lists (overrides or segment rules) | Large list lookup | Trim stale IDs, split lists, or move to rule-based inline segments or a custom attribute | +| Gate uses large ID lists (overrides or id based segments) | Large list lookup | Trim stale IDs, split lists, or move to conditional segments or a custom attribute | | Gate uses Regex or Version String operators | Expensive per-check parsing | Replace with exact matches or precomputed attributes | | Gate has many nested Passes Gate/Segment rules | Costs compound across nested evaluations | Reduce nesting or push logic into a single gate/segment | | Conditions have many values (50+) | Large value scans | Prune values, split cohorts, or move to a smaller segment | @@ -23,10 +23,9 @@ These labels are relative tiers, not fixed millisecond thresholds. Actual runtim ### Conditions -These costs are applied based on the **condition type** used in a gate. Only conditions with a significant impact are listed. If a condition is not listed, it does not significantly impact SDK evaluation speeds. +These evaluation costs are applied based on the **condition type** used in a gate. Only conditions with a significant impact are listed. If a condition is not listed, it does not significantly impact SDK evaluation speeds. -| Condition | Impact | worst case | Reason | -| | (assuming worst case) | | | +| Condition | Impact (assuming worst case) | worst case | Reason | | --------------- | --------------------- | ---------------------------------- | ---------------------------------------------------------------------------- | | BROWSER | medium | user agent | Requires user agent parsing unless Browser is explicitly provided | | BROWSER VERSION | medium | user agent | Requires user agent parsing unless Browser Version is explicitly provided | @@ -38,7 +37,9 @@ These costs are applied based on the **condition type** used in a gate. Only con | PASSES GATE | high | deeply nested with slow gates | complexity dependent on cost of nested gates | | FAILS GATE | high | deeply nested with slow gates | complexity dependent on cost of nested gates | + Tips: + Tips: + Tips: + Date: Wed, 18 Feb 2026 14:57:22 -0800 Subject: [PATCH 4/6] fix typo --- messages/evaluationScoring.mdx | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/messages/evaluationScoring.mdx b/messages/evaluationScoring.mdx index 90d533901..e5404cd29 100644 --- a/messages/evaluationScoring.mdx +++ b/messages/evaluationScoring.mdx @@ -39,7 +39,7 @@ These evaluation costs are applied based on the **condition type** used in a gat Tips: - - If you target OS/Browser or their versions, pass explicit values on `StatsigUser` to avoid user-agent parsing. Server SDKs support `os_name`, `os_version`, `browser_name`, and `browser_version`; client SDKs use `systemName`, `systemVersion`, `browserName`, and `browserVersion`. - If you target Country, pass `country` directly to avoid IP-based lookup. @@ -60,7 +60,7 @@ ID lists have a direct impact on evaluation performance. This applies both to ** Tips: - - Trim stale IDs and scope lists to active cohorts. - Split large lists into smaller, purpose-built cohorts. @@ -76,7 +76,9 @@ These costs are applied based on the **operator** used in a condition. If an ope | Version String | Medium | Requires semantic version comparison | | Array Contains Any / None / All | Low | Requires array scan | + Tips: + - **Match Regex String**: Replace with `ANY OF` or precompute a boolean/enum attribute. - **Version String**: Precompute a normalized numeric value or `major`/`minor` fields and compare those instead. - **Array Contains**: Keep arrays short and consider flattening frequent checks into a single boolean attribute. @@ -94,6 +96,6 @@ These costs are applied based on the **number of values** provided to a conditio Tips: - - If you are in the 50+ range, prune stale values or split cohorts. - Consider moving large lists into a smaller inline segment or a custom attribute computed upstream. From 312d0a1e184fc6b63327ee8e705f5836b9a658d9 Mon Sep 17 00:00:00 2001 From: araf-statsig Date: Wed, 18 Feb 2026 15:18:44 -0800 Subject: [PATCH 5/6] fix tag --- messages/evaluationScoring.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/messages/evaluationScoring.mdx b/messages/evaluationScoring.mdx index e5404cd29..eca7350c5 100644 --- a/messages/evaluationScoring.mdx +++ b/messages/evaluationScoring.mdx @@ -2,7 +2,7 @@ title: Evaluation Cost Breakdown --- -Feature gate evaluation speeds are determined by the overall complexity of all rules in a gate. Each **condition**, **operator**, and **value count** contributes to a score that classifies rules and gates as Normal, Slow, or Very Slow. Use this page to quickly identify the likely cause of a slow gate and the fastest way to fix it. +Feature gate evaluation speeds are determined by the overall complexity of all rules in a gate. Each **condition**, **operator**, and **value count** contributes to a score that classifies rules and gates as Normal, Slow, or Very Slow. The sum of the complexity of all rules equls the complexity of a gate. It is possible to not have a slow rule but a slow gate if all of the rules combined put the gate over the threshold. Use this page to quickly identify the likely cause of a slow gate and the fastest way to fix it. ## How to Interpret "Slow" and "Very Slow" @@ -39,7 +39,7 @@ These evaluation costs are applied based on the **condition type** used in a gat Tips: - + - If you target OS/Browser or their versions, pass explicit values on `StatsigUser` to avoid user-agent parsing. Server SDKs support `os_name`, `os_version`, `browser_name`, and `browser_version`; client SDKs use `systemName`, `systemVersion`, `browserName`, and `browserVersion`. - If you target Country, pass `country` directly to avoid IP-based lookup. @@ -60,7 +60,7 @@ ID lists have a direct impact on evaluation performance. This applies both to ** Tips: - + - Trim stale IDs and scope lists to active cohorts. - Split large lists into smaller, purpose-built cohorts. @@ -78,7 +78,7 @@ These costs are applied based on the **operator** used in a condition. If an ope Tips: - + - **Match Regex String**: Replace with `ANY OF` or precompute a boolean/enum attribute. - **Version String**: Precompute a normalized numeric value or `major`/`minor` fields and compare those instead. - **Array Contains**: Keep arrays short and consider flattening frequent checks into a single boolean attribute. @@ -96,6 +96,6 @@ These costs are applied based on the **number of values** provided to a conditio Tips: - + - If you are in the 50+ range, prune stale values or split cohorts. - Consider moving large lists into a smaller inline segment or a custom attribute computed upstream. From d8edf99d2827a17fe7c03d2758f4de43aa072291 Mon Sep 17 00:00:00 2001 From: araf-statsig Date: Wed, 18 Feb 2026 15:26:44 -0800 Subject: [PATCH 6/6] move tips inside tooltip --- messages/evaluationScoring.mdx | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/messages/evaluationScoring.mdx b/messages/evaluationScoring.mdx index eca7350c5..d70ba1cde 100644 --- a/messages/evaluationScoring.mdx +++ b/messages/evaluationScoring.mdx @@ -2,11 +2,11 @@ title: Evaluation Cost Breakdown --- -Feature gate evaluation speeds are determined by the overall complexity of all rules in a gate. Each **condition**, **operator**, and **value count** contributes to a score that classifies rules and gates as Normal, Slow, or Very Slow. The sum of the complexity of all rules equls the complexity of a gate. It is possible to not have a slow rule but a slow gate if all of the rules combined put the gate over the threshold. Use this page to quickly identify the likely cause of a slow gate and the fastest way to fix it. +Feature gate evaluation speeds are determined by the overall complexity of all rules in a gate. Each **condition**, **operator**, and **value count** contributes to a score that classifies rules and gates as Normal, Slow, or Very Slow. The sum of the complexity of all rules equls the complexity of a gate. It is possible to not have a slow rule but a slow gate if all of the rules combined, put the gate over the threshold. Use this page to quickly identify the likely cause of a slow gate and the fastest way to fix it. ## How to Interpret "Slow" and "Very Slow" -These labels are relative tiers, not fixed millisecond thresholds. Actual runtime depends on device/server, network, and whether the SDK is using the worst case scenario. (ex. even if country is included in the user object we assume the worst case scenario and that it is being ip geolocated). These labels are intended to be a warning. If you are using the best possible practice for your context, you can ignore them. +These labels are relative tiers, not fixed millisecond thresholds. Actual runtime depends on device/server, network, and whether the SDK is using the worst case scenario. (ex. even if country is included in the user object we assume the worst case scenario and that it is being IP geolocated). These labels are intended to be a warning. If you are using the best possible practice for your context, you can ignore them. ## Quick Diagnosis @@ -14,7 +14,7 @@ These labels are relative tiers, not fixed millisecond thresholds. Actual runtim | ---------------------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | Gate targets OS/Browser/Version and is slow | User-agent parsing at evaluation time | Pass `os_name`, `os_version`, `browser_name`, `browser_version` (server) or `systemName`, `systemVersion`, `browserName`, `browserVersion` (client) directly on `StatsigUser` | | Gate targets Country and is slow | IP geolocation lookup | Pass `country` directly on `StatsigUser` | -| Gate uses large ID lists (overrides or id based segments) | Large list lookup | Trim stale IDs, split lists, or move to conditional segments or a custom attribute | +| Gate uses large ID lists (overrides or ID based segments) | Large list lookup | Trim stale IDs, split lists, or move to conditional segments or a custom attribute | | Gate uses Regex or Version String operators | Expensive per-check parsing | Replace with exact matches or precomputed attributes | | Gate has many nested Passes Gate/Segment rules | Costs compound across nested evaluations | Reduce nesting or push logic into a single gate/segment | | Conditions have many values (50+) | Large value scans | Prune values, split cohorts, or move to a smaller segment | @@ -39,9 +39,9 @@ These evaluation costs are applied based on the **condition type** used in a gat Tips: + - If you target OS/Browser or their versions, pass explicit values on `StatsigUser` to avoid user-agent parsing. Server SDKs support `os_name`, `os_version`, `browser_name`, and `browser_version`; client SDKs use `systemName`, `systemVersion`, `browserName`, and `browserVersion`. + - If you target Country, pass `country` directly to avoid IP-based lookup. -- If you target OS/Browser or their versions, pass explicit values on `StatsigUser` to avoid user-agent parsing. Server SDKs support `os_name`, `os_version`, `browser_name`, and `browser_version`; client SDKs use `systemName`, `systemVersion`, `browserName`, and `browserVersion`. -- If you target Country, pass `country` directly to avoid IP-based lookup. @@ -49,7 +49,7 @@ For conditions that depend on other gates or segments, their evaluation cost is -### ID List Size (Overrides and Segment Conditions) +### ID List Size (Overrides and ID based Segments) ID lists have a direct impact on evaluation performance. This applies both to **ID list overrides** and **segment conditions** that reference ID lists. @@ -60,9 +60,10 @@ ID lists have a direct impact on evaluation performance. This applies both to ** Tips: + - Trim stale IDs and scope lists to active cohorts. + - Split large lists into smaller, purpose-built cohorts. -- Trim stale IDs and scope lists to active cohorts. -- Split large lists into smaller, purpose-built cohorts. + ### Operator @@ -78,10 +79,11 @@ These costs are applied based on the **operator** used in a condition. If an ope Tips: + - **Match Regex String**: Replace with `ANY OF` or precompute a boolean/enum attribute. + - **Version String**: Precompute a normalized numeric value or `major`/`minor` fields and compare those instead. + - **Array Contains**: Keep arrays short and consider flattening frequent checks into a single boolean attribute. -- **Match Regex String**: Replace with `ANY OF` or precompute a boolean/enum attribute. -- **Version String**: Precompute a normalized numeric value or `major`/`minor` fields and compare those instead. -- **Array Contains**: Keep arrays short and consider flattening frequent checks into a single boolean attribute. + ### Value Count @@ -96,6 +98,7 @@ These costs are applied based on the **number of values** provided to a conditio Tips: + - If you are in the 50+ range, prune stale values or split cohorts. + - Consider moving large lists into a smaller inline segment or a custom attribute computed upstream. -- If you are in the 50+ range, prune stale values or split cohorts. -- Consider moving large lists into a smaller inline segment or a custom attribute computed upstream. +