Skip to content

Speed up cold /score requests and ranking#87

Merged
adrunkhuman merged 21 commits into
masterfrom
perf/investigate-score-latency
Apr 19, 2026
Merged

Speed up cold /score requests and ranking#87
adrunkhuman merged 21 commits into
masterfrom
perf/investigate-score-latency

Conversation

@adrunkhuman
Copy link
Copy Markdown
Owner

@adrunkhuman adrunkhuman commented Apr 19, 2026

Summary

Speed up cold /score requests by tightening the scoring and ranking hot paths, while keeping the external response contract unchanged.

What Changed

  1. Allow up to two in-flight /score requests per worker instead of fully serializing them.
  2. Expand /score logging with cache status and scoring/ranking phase breakdowns so cold-path bottlenecks are measurable.
  3. Rework score_climate_matrix(...) to reuse buffers across the temperature, rain, sunshine, combine, and normalize stages.
  4. Rework indexed city ranking to reuse precomputed trig/population data and cheaper winner/penalty updates.
  5. Shrink the diversified ranking pool to reduce cold ranking cost after benchmarking the tradeoff on this branch.
  6. Remove the temporary /test profiling endpoint before merge.

Notes

  1. /score, /heatmap, and /probe response shapes stay contract-compatible.
  2. Ranked city composition may shift slightly because the ranking candidate pool is tighter than before.
  3. The branch keeps the new timing instrumentation because it was needed to drive and validate the optimizations.

Validation

  1. uv run pytest tests/test_app_shell.py tests/test_score_service.py tests/test_scoring.py tests/test_cities.py tests/test_heatmap.py -q
  2. uv run ruff check backend/main.py backend/score_service.py backend/scoring.py backend/cities.py tests/test_app_shell.py tests/test_score_service.py tests/test_scoring.py tests/test_cities.py
  3. uv run ty check backend tests/test_app_shell.py tests/test_score_service.py tests/test_scoring.py tests/test_cities.py tests/test_heatmap.py
  4. Repeated preview benchmark runs against the uncached profile suite via the temporary /test harness before it was removed:
    • score-only suite improved from roughly 6936ms to roughly 4423ms
    • score+heatmap suite improved from roughly 12222ms to roughly 10154ms

Closes #88

@adrunkhuman adrunkhuman changed the title Instrument /score cache states and ranking steps Speed up cold /score requests and ranking Apr 19, 2026
@adrunkhuman adrunkhuman merged commit 06a1d87 into master Apr 19, 2026
1 check passed
@adrunkhuman adrunkhuman deleted the perf/investigate-score-latency branch April 19, 2026 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate /score latency, ranking cost, and semaphore policy

1 participant