feat: add benchmarks and optimize acquire_job#17
Merged
Conversation
Add benchmark scripts for add-jobs throughput and acquire-job latency at 10k/100k/1M table sizes. Results are auto-written to BENCHMARKS.md with hardware info and library version. Add indexes on (run_after) and (pattern, run_after) to support acquire queries. Rewrite acquire_job/acquire_jobs as PL/pgSQL with separate code paths and dynamic SQL (EXECUTE) to prevent generic plan caching, bringing acquire latency from ~110ms to ~0.5ms at 1M rows.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add automated benchmark suites for add-jobs throughput and acquire-job latency at scale (10k/100k/1M rows). Results are auto-written to BENCHMARKS.md with hardware and version info.
Optimize acquire_job and acquire_jobs performance from ~110ms to ~0.5ms at 1M rows by: (1) adding indexes on (run_after) and (pattern, run_after), (2) splitting PL/pgSQL into separate code paths to eliminate OR conditions that block index usage, (3) using EXECUTE (dynamic SQL) instead of RETURN QUERY to prevent generic plan caching.
New scripts:
bun run bench,bun run bench:add,bun run bench:acquire.🤖 Generated with Claude Code