Skip to content

cache: reduce contention in load_or_fetch#20

Merged
blackopsrepl merged 1 commit intomainfrom
issue/7-reduce-load-or-fetch-contention
Mar 21, 2026
Merged

cache: reduce contention in load_or_fetch#20
blackopsrepl merged 1 commit intomainfrom
issue/7-reduce-load-or-fetch-contention

Conversation

@blackopsrepl
Copy link
Copy Markdown
Contributor

Summary

load_or_fetch was still holding the global cache write lock across disk checks, cache-file reads, Overpass fetches, and cache-file writes. That serialized unrelated regions and also left same-key duplicate work underspecified.

This change refactors the load path to use a double-checked insert with per-key in-flight deduplication:

  • slow disk and network I/O now runs outside the global cache write lock
  • concurrent callers for different cache keys can progress independently
  • concurrent callers for the same cache key share one in-flight load and reuse the inserted result
  • focused async tests cover both concurrency cases

Validation

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test

Fixes #7

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: df8d319441

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +492 to +495
let should_remove = in_flight
.get(cache_key)
.map(|current| Arc::ptr_eq(current, slot) && Arc::strong_count(slot) == 2)
.unwrap_or(false);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Drop _slot_guard before pruning IN_FLIGHT_LOADS

At both cleanup call sites in load_or_insert(), _slot_guard and slot are still alive, so Arc::strong_count(slot) is at least 3 here (the map entry, the local slot, and the OwnedMutexGuard). That makes should_remove permanently false, so every distinct cache_key leaves an Arc<Mutex<()>> behind in IN_FLIGHT_LOADS even after the load completes. In a long-lived service that sees many one-off bounding boxes, this side map will grow without bound.

Useful? React with 👍 / 👎.

@blackopsrepl blackopsrepl merged commit 2580b04 into main Mar 21, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cache: reduce contention in load_or_fetch by removing write lock from slow I/O paths

1 participant