Skip to content

feat(cache): add L2 disk cache for original fetched files (HTTP/S3)#28

Merged
nakamuraos merged 1 commit into
mainfrom
feat/original-cacheable
Apr 9, 2026
Merged

feat(cache): add L2 disk cache for original fetched files (HTTP/S3)#28
nakamuraos merged 1 commit into
mainfrom
feat/original-cacheable

Conversation

@nakamuraos
Copy link
Copy Markdown
Contributor

Summary

  • Adds OriginCache that stores raw fetched bytes (before transforms) to disk under <cache_dir>/origin/,
    keyed by SHA-256 of the source URL
  • On cache miss for a transformed request, checks origin cache first - if the source URL was previously
    fetched, skips the origin fetch and runs transforms on cached bytes
  • Supports per-entry TTL: HTTP streaming path extracts Cache-Control: max-age / Expires headers from
    origin response; buffered path falls back to PP_CACHE_DISK_TTL_SECS
  • Applies to HTTP and S3 sources only; local and alias sources are excluded

Test Plan

  • cargo test - 396 tests pass
  • Request same URL with different transform params - second request should skip origin fetch (check logs
    for "origin cache hit")
  • Verify <cache_dir>/origin/ directory is populated after first request
  • Verify TTL-expired entries are cleaned up on next cleanup cycle

@nakamuraos nakamuraos merged commit ff2bff8 into main Apr 9, 2026
4 checks passed
@nakamuraos nakamuraos deleted the feat/original-cacheable branch April 9, 2026 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant