Skip to content

Conversation

@YuriZmytrakov
Copy link
Collaborator

This PR implements a CQL2 Abstract Syntax Tree (AST) structure for CQL2 Filters to enable better parameter access for index selection in search queries:

  • Introduces a structured AST representation of CQL2 queries (replacing dict-based processing)
  • Adds datetime extraction logic to easily identify and access datetime parameters
  • Provides ability for selecting appropriate indexes such as datetime, start_datetime, end_datetime
    The AST-based approach makes it easier to parse CQL2 filters, particularly for identifying datetime constraints (datetime, start_datetime, end_datetime) to optimize search performance through proper index selection.

Examples of AST nodes:
AND:

LogicalNode(op=<LogicalOp.AND: 'and'>, children=[ComparisonNode(op=<ComparisonOp.EQ: '='>, field='collection', value='landsat-8-l1'), 
                                                 ComparisonNode(op=<ComparisonOp.GT: '>'>, field='properties.cloud_cover', value=50)])

OR:

LogicalNode(op=<LogicalOp.OR: 'or'>, children=[ComparisonNode(op=<ComparisonOp.EQ: '='>, field='properties.platform', value='landsat-8'), 
                                               ComparisonNode(op=<ComparisonOp.EQ: '='>, field='properties.platform', value='sentinel-2')])

NOT:

LogicalNode(op=<LogicalOp.NOT: 'not'>, children=[ComparisonNode(op=<ComparisonOp.EQ: '='>, field='properties.cloud_cover', value=0)])

EQ (=):

ComparisonNode(op=<ComparisonOp.EQ: '='>, field='properties.cloud_cover', value=20)

NEQ (<>)

ComparisonNode(op=<ComparisonOp.NEQ: '<>'>, field='properties.platform', value='landsat-8')

LT (<)

ComparisonNode(op=<ComparisonOp.LT: '<'>, field='properties.cloud_cover', value=30)

LTE (<=)

ComparisonNode(op=<ComparisonOp.LTE: '<='>, field='properties.cloud_cover', value=50)

GT (>)

ComparisonNode(op=<ComparisonOp.GT: '>'>, field='properties.gsd', value=10)

GTE (>=)

ComparisonNode(op=<ComparisonOp.GTE: '>='>, field='properties.cloud_cover', value=0)

IS_NULL

ComparisonNode(op=<ComparisonOp.IS_NULL: 'isNull'>, field='properties.cloud_cover', value=None)

LIKE

AdvancedComparisonNode(op=<AdvancedComparisonOp.LIKE: 'like'>, field='id', value='LC08%')

BETWEEN

AdvancedComparisonNode(op=<AdvancedComparisonOp.BETWEEN: 'between'>, field='properties.cloud_cover', value=(10, 50))

IN

AdvancedComparisonNode(op=<AdvancedComparisonOp.IN: 'in'>, field='properties.platform', value=['landsat-8', 'sentinel-2', 'modis'])

S_INTERSECTS

SpatialNode(op=<SpatialOp.S_INTERSECTS: 's_intersects'>, field='geometry', geometry={'type': 'Polygon', 'coordinates': [[[-110, 40], [-110, 41], [-109, 41], [-109, 40], [-110, 40]]]})

S_CONTAINS

SpatialNode(op=<SpatialOp.S_CONTAINS: 's_contains'>, field='geometry', geometry={'type': 'Point', 'coordinates': [-110.5, 40.5]})

S_WITHIN

SpatialNode(op=<SpatialOp.S_WITHIN: 's_within'>, field='geometry', geometry={'type': 'Polygon', 'coordinates': [[[-120, 35], [-120, 45], [-100, 45], [-100, 35], [-120, 35]]]})

S_DISJOINT

SpatialNode(op=<SpatialOp.S_DISJOINT: 's_disjoint'>, field='geometry', geometry={'type': 'Polygon', 'coordinates': [[[0, 0], [0, 10], [10, 10], [10, 0], [0, 0]]]})

DateTimeRange

LogicalNode(op=<LogicalOp.AND: 'and'>, children=[ComparisonNode(op=<ComparisonOp.GTE: '>='>, field='properties.datetime', value='2024-01-01T00:00:00Z'), 
                                                 ComparisonNode(op=<ComparisonOp.LTE: '<='>, field='properties.datetime', value='2024-12-31T23:59:59Z')])

DateTimeExactNode

ComparisonNode(op=<ComparisonOp.EQ: '='>, field='properties.datetime', value='2024-06-15T10:30:00Z')

PR Checklist:

  • Code is formatted and linted (run pre-commit run --all-files)
  • Tests pass (run make test)
  • Documentation has been updated to reflect changes, if applicable
  • Changes are added to the changelog

@YuriZmytrakov YuriZmytrakov force-pushed the CAT-1447 branch 5 times, most recently from 99366c8 to 2279c72 Compare December 15, 2025 14:50
- Added Abstract Syntax Tree (AST) representation for CQL2 queries replacing dictionary-based queries
- Implemented datetime extraction logic to identify datetime, start_datetime, and end_datetime
- Support for complex queries with logical operators, spatial operations, and property comparisons
- Improved query parsing and maintainability of CQL2-JSON filter requests
Yuri Zmytrakov added 2 commits December 15, 2025 16:04
- Added tests for CQL2-JSON datetime filter operators
- Tested datetime field operators: =, <=, >=, AND combinations
- Tested start_datetime and end_datetime field operators
- Verified range queries work correctly with date fields
- Ensured AST parsing correctly handles datetime comparisons
@YuriZmytrakov YuriZmytrakov changed the title feat: Rewrite CQL2 implementation for SFEOS Implement CQL2-JSON AST for datetime index selection Dec 15, 2025
@Gomez324
Copy link
Collaborator

Hey, I ran one test with this query and body:
127.0.0.1:8080/v1/search

{
  "filter-lang": "cql2-json",
  "filter": {
    "op": "and",
    "args": [
      {
        "op": "=",
        "args": [{ "property": "collection" }, "sentinel-2-l2a"]
      },
      {
        "op": ">=",
        "args": [{ "property": "datetime" }, "2015-01-01T00:00:00Z"]
      },
      {
        "op": "<=",
        "args": [{ "property": "datetime" }, "2024-12-31T23:59:59Z"]
      }
    ]
  }
}

And I got that index_param has this value. it looks like it's not selecting temporal indices. Please add tests for temporal indices.

(Pdb) index_param
'items_*,-*kibana*,-collections*'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants