fix: chunk IDs in hydrateEntryBylines to avoid D1 SQL variable limit#242
fix: chunk IDs in hydrateEntryBylines to avoid D1 SQL variable limit#242JiayuuWang wants to merge 1 commit intoemdash-cms:mainfrom
Conversation
Cloudflare D1 enforces a bound-parameter limit lower than SQLite's default 999. When getEmDashCollection returns a large result set, hydrateEntryBylines passes all entry IDs in a single call to getBylinesForEntries, which issues unbounded IN (...) clauses through getContentBylinesMany, getAuthorIds, and findByUserIds. This causes D1_ERROR: too many SQL variables on collections with 50+ entries, and byline data is silently dropped. Fix: chunk IDs into batches of 50 before calling getBylinesForEntries, then merge the result maps. 50 is conservative enough to stay within D1's limit while keeping round-trips low. Fixes emdash-cms#219 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
I have read the CLA Document and I hereby sign the CLA You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
|
Thanks for the contribution. This has already been fixed in #223 so I'm closing it. |
Summary
Fixes #219.
Cloudflare D1 enforces a SQL bound-parameter limit lower than SQLite's default 999. When
getEmDashCollectionreturns a large result set,hydrateEntryBylinespassed all entry IDs in one call togetBylinesForEntries, which builds unboundedIN (?, ?, …)clauses across three code paths:BylineRepository.getContentBylinesMany—WHERE cb.content_id IN (…)getAuthorIdsfallback —WHERE id IN (…)BylineRepository.findByUserIds—WHERE user_id IN (…)On collections with 50+ entries this triggers
D1_ERROR: too many SQL variables, and byline data is silently dropped (caught by the existingtry/catch).Fix
Chunk the ID array in
hydrateEntryBylinesinto batches of 50 before callinggetBylinesForEntries, then merge the result maps. This is the single control point for all three downstream queries. A batch size of 50 stays well within D1's limit while keeping round-trips low.Test plan
getEmDashCollection('posts', { limit: 100 })— should succeed with byline data hydratedD1_ERROR: too many SQL variableserrors in the Worker logs🤖 Generated with Claude Code