Skip to content

Auto-run rowCount backfill during make dev #95

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
urugondavishnu wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Auto-run rowCount backfill during make dev #95

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a796bab
Auto-run rowCount backfill during make dev
urugondavishnu May 26, 2026
a8c102d
Merge branch 'main' into auto-backfill-row-counts-in-dev
MMeteorL Jun 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion makefiles/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
SHELL := /usr/bin/env bash

.PHONY: all dev validate-dev-env down clean convex-push convex-env seed-public-datasets ensure-admin-key install-deps
.PHONY: all dev validate-dev-env down clean convex-push convex-env seed-public-datasets backfill-row-counts ensure-admin-key install-deps

all: dev

Expand All @@ -20,6 +20,7 @@ dev: validate-dev-env install-deps
$(MAKE) ensure-admin-key
$(MAKE) convex-env
$(MAKE) convex-push
$(MAKE) backfill-row-counts
docker compose -f docker-compose.dev.yml up --build -d
@echo ""
@echo "BigSet is ready!"
Expand Down Expand Up @@ -146,6 +147,20 @@ seed-public-datasets:
@test -f .env || { echo "Error: .env not found. Run: cp .env.example .env"; exit 1; }
@cd frontend && node ../scripts/with-root-env.mjs npx convex run publicSeed:seedPublicDatasets

# One-shot idempotent migration: ensure every dataset doc has an accurate
# `rowCount` field (the field the dashboard card footer reads). Hooked
# into `make dev` so contributors pulling a branch that adds the field
# don't have to remember to run it manually. Re-running on a fully
# migrated DB reports `patched: 0, alreadyCorrect: N`.
#
# Matches the `seed-public-datasets` invocation shape (no --admin-key flag).
# `with-root-env.mjs` injects CONVEX_SELF_HOSTED_ADMIN_KEY into the env so
# the convex CLI picks it up automatically — avoids the Windows-cmd issue
# where the `|` in the admin key gets parsed as a shell pipe.
backfill-row-counts:
@test -f .env || { echo "Error: .env not found. Run: cp .env.example .env"; exit 1; }
@cd frontend && node ../scripts/with-root-env.mjs npx convex run datasets:backfillRowCounts

down:
docker compose -f docker-compose.dev.yml down

Expand Down