IBX-11146: Updated filtering logic to use LanguageCode criterion with {terms} clause#107
IBX-11146: Updated filtering logic to use LanguageCode criterion with {terms} clause#107
Conversation
ccf7642 to
b96aee5
Compare
tests/lib/Search/Query/Common/CriterionVisitor/LanguageCodeInTest.php
Outdated
Show resolved
Hide resolved
composer.json
Outdated
| "ext-json": "*", | ||
| "ext-xmlwriter": "*", | ||
| "ibexa/core": "~4.6.0@dev", | ||
| "ibexa/core": "dev-ibx-11146-integration-test-for-solr-languages as 4.6.0@dev", |
There was a problem hiding this comment.
Friendly reminder before merge
There was a problem hiding this comment.
We need to switch to our composer-install action and use dependencies.json instead.
At this point I think this is more of a note to myself :D
alongosz
left a comment
There was a problem hiding this comment.
Nice one 💪 I was aware of these long queries for quite some time, but never came up with a proper solution 😅
composer.json
Outdated
| "ext-json": "*", | ||
| "ext-xmlwriter": "*", | ||
| "ibexa/core": "~4.6.0@dev", | ||
| "ibexa/core": "dev-ibx-11146-integration-test-for-solr-languages as 4.6.0@dev", |
There was a problem hiding this comment.
We need to switch to our composer-install action and use dependencies.json instead.
At this point I think this is more of a note to myself :D
718cd09 to
452418e
Compare
|
tomaszszopinski
left a comment
There was a problem hiding this comment.
QA approved on Ibexa DXP 4.6 ddev/solr9



Related PRs:
Description:
Problem
When many languages are configured in Ibexa (e.g., >20-30), the language fallback logic in
NativeCoreFiltergenerates an exponentially growing number of boolean clauses. This causes Solr queries to fail with amaxBooleanClauseserror (default limit is 1024).The problematic query structure for exclusion was:
NOT (content_language_codes_ms:"lang1" OR content_language_codes_ms:"lang2" ...)For a fallback chain, this exclusion is repeated for every priority level, leading to O(N^2) complexity in clause count.
Solution
Optimized the query generation to use Solr's Terms Query Parser (
{!terms}) for language exclusions. This parser treats a list of values as a single query clause, regardless of the number of elements.The new query structure is:
_query_:"{!terms f=content_language_codes_ms}lang1,lang2..."This reduces the complexity from O(N^2) boolean clauses to O(N) single clauses, effectively bypassing the
maxBooleanClauseslimit for this use case.Applies
strtolower()to values because the Terms parser bypasses analysis, but the underlying field (content_language_codes_ms) is lowercased in the schema.For QA:
Dont focus at the 50 languages thing - check if site access combinations languages work properly, take
alwaysAvailableflag into consideration, fallback, default languages and so on.Documentation: