Issue 469 mutable ttl (#508)

Author: chenziliang

docs/materialized-view.md

@@ -67,7 +67,7 @@ AS
  [COMMENT '<comments>']
 ```
 
-Example
+### Example
 
 ```sql
 -- Create a source stream
@@ -97,7 +97,7 @@ SELECT * FROM aggr_results;
 
 :::info
 A **Materialized View** is always bound to a **streaming `SELECT` query**.
-If you want to materialize a **historical query** on a schedule, consider using a [Timeplus scheduled task](/task) instead.
+If you want to materialize a **historical query** on a schedule, consider using a [**Timeplus scheduled task**](/task) instead.
 :::
 
 ### Settings

docs/mutable-stream-ttl.md

@@ -0,0 +1,104 @@
+# TTL
+
+TTL for Mutable Streams works differently from [Append Stream](/append-stream-ttl) because data is row-encoded.
+
+Mutable Streams evaluate the timestamp of each row in the historical store asynchronously during background data merge or compaction of the key space. Expired rows are automatically garbage-collected when their TTL elapses.
+
+:::info
+A Mutable Stream created without a TTL cannot be altered later to support TTL, and similarly, a stream with TTL cannot have TTL removed.
+:::
+
+## Retention Based on Wall Clock
+
+In this mode, you specify the storage setting `ttl_seconds` with a non-zero value.
+
+When a row is inserted, the system automatically assigns a **wall-clock timestamp** to it.
+If a row has not been updated for longer than `ttl_seconds`, it becomes **eligible for garbage collection** during background compaction.
+
+### Syntax
+
+```sql
+CREATE MUTABLE STREAM ...
+SETTINGS
+    ttl_seconds=<ttl-seconds>, ...
+```
+
+### Example
+
+```sql
+CREATE MUTABLE STREAM ttl_cached(
+    id string,
+    name string,
+    price string,
+    description string,
+    event_time datetime
+)
+PRIMARY key id
+SETTINGS
+    ttl_seconds = 3600;
+```
+
+In this example, each row is assigned a **TTL of 1 hour** (3600 seconds).
+If a row is not updated within that time window, it will be **automatically removed** during the next background compaction.
+
+## Retention Based on Event Timestamp
+
+In this mode, you must specify both the storage settings `ttl_seconds` (a non-zero value) and `ttl_column`.
+
+The specified **TTL column** represents the event timestamp of each row. During background compaction, the system evaluates this column to determine whether the row has expired and should be garbage-collected.
+
+### Syntax
+
+```sql
+CREATE MUTABLE STREAM ...
+SETTINGS
+    ttl_seconds=<ttl-seconds>,
+    ttl_column=<ttl-column>, ...
+```
+
+### Example
+
+```sql
+CREATE MUTABLE STREAM event_ttl_cached(
+    id string,
+    name string,
+    price string,
+    description string,
+    event_time datetime
+)
+PRIMARY key id
+SETTINGS
+    ttl_seconds = 3600,
+    ttl_column = 'event_time';
+```
+
+In this example, the stream is configured with a **TTL of 1 hour**.
+If `now() - event_time >= 3600`, the row is considered **expired** and will be **garbage-collected** during background compaction.
+
+## Controlling TTL Frequency
+
+TTL evaluation occurs during background **merge** or **compaction** operations.
+You can control how frequently these compactions run by setting `periodic_compaction_seconds` in `kvstore_options`.
+
+**Example:**
+
+```
+CREATE MUTABLE STREAM ...
+SETTINGS
+    ttl_seconds = <ttl-seconds>,
+    ttl_column = <ttl-column>,
+    kvstore_options = 'periodic_compaction_seconds=1800;';
+```
+
+In this example, the storage engine is configured to trigger periodic compaction every **30 minutes** (1800 seconds), which indirectly determines how often expired rows are cleaned up.
+
+## Manually Triggering Compaction
+
+Expired data is removed only during background compaction.
+If you want to accelerate TTL cleanup, you can manually trigger compaction using the `OPTIMIZE` command.
+
+```sql
+OPTIMIZE STREAM <db.stream-name>;
+```
+
+This command initiates an immediate, unscheduled merge or compaction for the specified stream, helping reclaim space and remove expired rows sooner.

docs/mutable-stream.md

@@ -40,6 +40,7 @@ SETTINGS
     logstore_retention_bytes=<retention-bytes>,
     logstore_retention_ms=<retention-ms>,
     ttl_seconds=<ttl-seconds>,
+    ttl_column=<ttl-column>,
     auto_cf=[true|false],
     placement_policies='<placement-policies>',
     late_insert_overrides=[true|false],
@@ -58,23 +59,35 @@ SETTINGS
     enable_statistics=[true|false];
 ```
 
+### Example
+
+```sql
+CREATE MUTABLE STREAM products(
+  id string,
+  name string,
+  price float32,
+  description string
+)
+PRIAMRY KEY id;
+```
+
 ### Storage Architecture
 
 Each shard in a Mutable Stream has [dural storage](/architecture#dural-storage), consisting of:
 
 - Write-Ahead Log (WAL), powered by NativeLog. Enabling incremental processing.
 - Historical key-value store, powered by RocksDB.
 
-Data is first ingested into the WAL, and then asynchronously committed to the row store in large batches.  
+Data is first ingested into the WAL, and then asynchronously committed to the row store in large batches.
 
 The Mutable Stream settings allow fine-tuning of both storage layers to balance performance, durability, and efficiency.
 
 ### PRIMARY KEY
 
-**PRIMARY KEY** — Defines the uniqueness of a row in a Mutable Stream. **Required.**  
+**PRIMARY KEY** — Defines the uniqueness of a row in a Mutable Stream. **Required.**
 
-Rows are organized and sorted based on the primary key, and the primary index is built on top of it.  
-See [Mutable Stream Indexes](/mutable-stream-indexes) for more details.  
+Rows are organized and sorted based on the primary key, and the primary index is built on top of it.
+See [Mutable Stream Indexes](/mutable-stream-indexes) for more details.
 
 ### Secondary Indexes
 
@@ -139,10 +152,16 @@ Garbage collection runs periodically in the background (default: every 5 minutes
 
 #### `ttl_seconds`
 
-Retention policy for the **historical key-value store (RocksDB)** based on ingest time (wall clock).  When a row exceeds this threshold, it is eligible for deletion. Garbage collection is background and **non-deterministic**, so do not rely on exact deletion timing.
+Refer [Mutable Stream TTL](/mutable-stream-ttl) for details.
 
 **Default**: `-1` (no retention)
 
+#### `ttl_column`
+
+Refer [Mutable Stream TTL](/mutable-stream-ttl) for details.
+
+**Default**: `""`
+
 #### `auto_cf`
 
 Automatically groups columns of similar type/characteristics into column families.

sidebars.js

@@ -354,6 +354,11 @@ const sidebars = {
                   id: "mutable-stream-coalesced",
                   label: "Coalesced Mutable Stream",
                 },
+                {
+                  type: "doc",
+                  id: "mutable-stream-ttl",
+                  label: "TTL",
+                },
               ]
             },
             {