fix(src): Fix 24h price gap calculation by widening retention and lookup window#300
fix(src): Fix 24h price gap calculation by widening retention and lookup window#300aristidesstaffieri merged 5 commits intomainfrom
Conversation
The Redis Time Series retention period (24h) and the 24h-ago lookup threshold (5min tolerance) created only a 5-minute overlap window for finding historical price data. As the token count grows and update cycles take longer, this narrow window frequently contains no data points, causing percentagePriceChange24h to return null. - Increase RETENTION_PERIOD from 24h to 25h - Increase DEFAULT_ONE_DAY_THRESHOLD_MS from 5min (300000ms) to 30min (1800000ms) This expands the overlap window from 5 minutes to 90 minutes, tolerating longer update cycles and transient Horizon outages without losing 24h price change data.
There was a problem hiding this comment.
Pull request overview
This PR aims to reduce null 24h price-change results by increasing the Redis TimeSeries retention period and widening the time window used to find the “~24h ago” historical price point.
Changes:
- Increased Redis TimeSeries retention from 24h to 25h.
- Increased the default “one day threshold” delta from 5 minutes to 30 minutes.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
piyalbasu
left a comment
piyalbasu
left a comment
There was a problem hiding this comment.
This change makes sense to me. The Copilot comments are reasonable to add (they're mostly around documentation and updating tests)
Wondering if this new window size is based on a metric or if this is more of a "let's make it a little bit bigger and see if it works"
The 24h price change calculation frequently returns null because a single threshold (oneDayThreshold) was used for both the minimum history requirement and the revRange lookup window. With a 5-minute tolerance and 24h retention, the overlap window for finding a historical price point was only 5 minutes — easily missed during update gaps. - Increase RETENTION_PERIOD from 24h to 25h for wider data retention - Add separate PRICE_LOOKUP_TOLERANCE_MS (30min) for the revRange lookup - Keep DEFAULT_ONE_DAY_THRESHOLD_MS (5min) for the strict ~24h history gate The history gate still requires ~23h55m of data before computing a 24h change (no stale data). The lookup now searches with a 90-minute overlap window instead of 5 minutes, tolerating gaps in price updates., wiring up the handoff from setup to live ingestion
piyalbasu
left a comment
piyalbasu
left a comment
There was a problem hiding this comment.
@aristidesstaffieri mentioned in Slack: I spot checked a few and saw that when they miss their window it wasn't by much, 62 seconds in one case. So this should give it enough wiggle room to limp along for now.
Given that, this PR lgtm!
I spot checked a few and saw that when they miss their window it wasn't by much, 62 seconds in one case. It varies case by case so I tried to find a value that gave it enough wiggle room to work around the variance without going overboard. |
Add tests verifying that the 24h history gate and the lookup tolerance
operate independently:
- Token with 23h54m of history (1min under gate) returns null for
percentagePriceChange24h and does not attempt the revRange lookup
- Token with 23h56m of history (1min over gate) computes the 24h change
using the wider 30min PRICE_LOOKUP_TOLERANCE_MS, not the strict 5min
DEFAULT_ONE_DAY_THRESHOLD_MS
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…300 follow-up) Prevents SSRF by ignoring user-provided network_url in /simulate-tx and /submit-tx, resolving the RPC server from network_passphrase instead. The schema still accepts network_url for backwards compatibility but the value is no longer read. Adds tests for both routes covering the new behavior.
…300 follow-up) Prevents SSRF by ignoring user-provided network_url in /simulate-tx and /submit-tx, resolving the RPC server from network_passphrase instead. The schema still accepts network_url for backwards compatibility but the value is no longer read. Adds tests for both routes covering the new behavior.
…303) * rc): remove network_url usage from simulate-tx and submit-tx routes (#300 follow-up) Prevents SSRF by ignoring user-provided network_url in /simulate-tx and /submit-tx, resolving the RPC server from network_passphrase instead. The schema still accepts network_url for backwards compatibility but the value is no longer read. Adds tests for both routes covering the new behavior. * chore: remove redundant network_url schema properties and tests AJV's removeAdditional:true already silently strips unknown fields, so explicitly listing network_url in the schema was unnecessary.
30 * 60 * 1000instead of magic number1800000RETENTION_PERIODcomment already updated to "25 hours" in remote branchPRICE_LOOKUP_TOLERANCE_MSvalue from1800000to30 * 60 * 1000and improve doc comment