We don't want to create unnecessarily large amount of requests on upstream servers. Therefore we probably need a cache for the scraper.
Our usual cache storage is ObjectStorage.
Before starting with implementation we should decide:
- Does a cache makes sense?
- What should be cache?
- How works cache invalidation?
We don't want to create unnecessarily large amount of requests on upstream servers. Therefore we probably need a cache for the scraper.
Our usual cache storage is ObjectStorage.
Before starting with implementation we should decide: