Feat/memory eviction#14
Merged
Merged
Conversation
KvEngine now maintains a lock-light AtomicU64 counter that sums key bytes, value bytes, and a fixed per-entry overhead. Every write path goes through a small set of helper methods (track_insert, track_remove, track_clear, untrack) so the counter stays consistent with the underlying HashMap without scattering accounting code across every command. The counter is surfaced through used_memory() and memory_usage(key), the latter replicating Redis' per-key MEMORY USAGE reply. No policy is enforced yet; this commit only introduces observability so that the follow-up maxmemory enforcement can reuse the same figures. indexmap is added as a dependency in preparation for the LRU tracker that lands next.
… policies Introduce a small storage::eviction module that owns the policy enum, the configuration struct, and the candidate-selection algorithm. KvEngine gains an Arc<RwLock<EvictionConfig>> so the policy can be reconfigured at runtime, and ValueEntry now carries a last_access timestamp so the approximate LRU policies have something to rank by. Writes (SET, SETNX, MSET, INCR family, APPEND) call enforce_memory_limit before mutating the store: the helper samples policy-eligible keys, picks a victim, and loops until the new entry fits under maxmemory. noeviction and volatile-without-TTL datasets surface FerrumError::OutOfMemory, which already maps to the Redis-style '-OOM' reply at the wire. Every eviction writes a DEL record so the AOF stays authoritative across restarts. GET and MGET refresh last_access so the LRU signal reflects read traffic too, matching Redis' approximate-LRU behaviour.
Add CLI flags (--maxmemory, --maxmemory-policy, --maxmemory-samples) and matching config-file directives, all plumbed through to KvEngine::set_eviction_config on startup. Byte sizes accept the Redis-style b/k/kb/m/mb/g/gb suffixes so operators can write maxmemory 100mb in either source. Wire up two observability commands: MEMORY USAGE key returns the approximate byte cost of a single entry (null bulk for missing keys), and INFO [section] renders the minimum subset operators expect - server/memory/keyspace. The memory section surfaces used_memory, maxmemory, the active policy, and the sample size, so a redis-cli INFO memory looks the way automation already parses. End-to-end coverage in tests/eviction_test.rs exercises every new wire contract plus a real LRU eviction path driven through the network layer.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.