Development#94
Merged
Merged
Conversation
Pivots the Top Movers widget from "who went up / down" to "markets departing from their normal trends". With ~29 keys and the polling density we have, "top N movers" can never be reliable; "unusual vs this item's own dispersion" can be, even on sparse data. Honest framing about what the data supports. What's in the box: - Flyway V1.24 + new item_unusual_candidates table. One row per (item_id, source) with the multi-horizon stats flattened across columns. Partial index on (source, unusualness_score DESC). - ItemUnusualCandidatesRepository.RebuildAsync — single SQL pass. Trimmed-median baseline (10/90 bounds, NOW-30d to NOW-1d, min 10 buckets); CV of daily medians for dispersion (min 14 days). Per horizon (1h / 6h / 24h / 7d): median + sample count + move + z, with min-sample thresholds 1/3/6/24 buckets respectively. Derived unusualness_score = max(|z|), dominant_horizon, direction. UPSERT. - Hangfire RebuildUnusualCandidates at minute 45 every 6h, after SummariseChangeLogs (00) and RebuildVolatilityStats (30). - GET /api/items/unusual?source=&limit=&minScore=. Default minScore 1.5σ, limit clamped 1..50. - Widget rewrite — three cards: Top risers, Top fallers, Unusual activity. Most-active card removed (saturation made it useless). Unusual rows show a server-formatted "↑ 3.4σ in last 24h vs month" string + the dominant horizon's window price. Frontend types + useUnusualItems hook follow the established useItemVolatility pattern. percentile_cont casts to numeric up- front (same ROUND-on-double lesson from V1.21). TODO.md + context/session-handoff.md updated to reflect the ship and list the deferred follow-ups (re-score on home-page load, mode-to-bucket display on item details, volatility-of-volatility, confidence chips, dropping legacy current_price/price_change_* columns). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…into development
SummariseChangeLogs was calling BuildSummariesAsync with the full [earliestChangeTime, NOW) window after V1.22 truncated the summaries table. With ~5 months of change_logs to roll up into 1h buckets (~8M target rows on a B1ms), the single UPSERT was exceeding the default 30s Npgsql command timeout and failing in prod. Two complementary fixes: - Chunk the orchestration in DatabaseService.SummariseChangeLogsAsync into 7-day windows. Each chunk finishes in seconds. Progress is checkpointed by GetLatestBucketStartAsync, so a Hangfire retry resumes from the last completed chunk rather than restarting. - Bump the per-call command timeout in BuildSummariesAsync to 10 minutes (vs the default 30s). Even with 7-day chunks this is defensive headroom for items × buckets aggregations on B1ms. Steady-state operation is unaffected: post-backfill, each scheduled run only has one chunk's worth of new data (the gap since the last run). The backfill machinery is only exercised after a truncate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Heading was unconditionally rendered in Home.tsx while TopMovers returned null when noData. After V1.22 truncated the summaries the section was a bald heading with empty space underneath. Move the heading + subtitle into TopMovers itself so the existing noData check governs everything together. Also refresh the subtitle wording — "Most-traded items" is stale now that the Most active card is gone; replaced with "Items whose prices moved most over the last 24 hours, alongside markets showing statistically unusual activity" to match the actual three cards. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 1.5σ z-floor, 10% absolute-move floor, and per-horizon min-sample thresholds are all one-line constants we'll likely want to tune once the new card has been live for a while. Captured in TODO.md so a future session knows where the knobs are.
…into development
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 459124b91b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+215
to
+217
| FROM scored s | ||
| ON CONFLICT (item_id, source) DO UPDATE SET | ||
| computed_at = EXCLUDED.computed_at, |
There was a problem hiding this comment.
This rebuild is insert/update-only, so rows for (item_id, source) pairs that no longer appear in scored are never removed. Once an item has no qualifying 30-day baseline (for example, it stops trading for a while), its old unusualness_score can remain in item_unusual_candidates indefinitely and still be returned by GetTopAsync, which corrupts the home-page ranking with stale results. Add a delete/truncate step (or otherwise mark-and-sweep by computed_at) as part of each rebuild cycle.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.