Skip to content

refactor: implement canonical env contract for staging/production switching#201

Merged
VibhavSetlur merged 16 commits into
ModelSEED:stagingfrom
VibhavSetlur:staging
May 11, 2026
Merged

refactor: implement canonical env contract for staging/production switching#201
VibhavSetlur merged 16 commits into
ModelSEED:stagingfrom
VibhavSetlur:staging

Conversation

@VibhavSetlur
Copy link
Copy Markdown
Collaborator

Environment System Refactor: Canonical Config with Deployment Mode

Summary

Comprehensive refactor of ModelSEED-UI environment variable system to support clean staging/production switching with Docker Compose. Implements canonical NEXT_PUBLIC_DEPLOYMENT_MODE with strict 3-level precedence model and eliminates confusing env var naming.

What's Changed

Core Configuration System

  • Canonical mode var: NEXT_PUBLIC_DEPLOYMENT_MODE (not ENV_MODE, DEPLOY_ENV, etc.)
  • 3-level precedence:
    1. Explicit override (highest priority) - e.g., NEXT_PUBLIC_API_BASE_URL
    2. Mode-specific default - e.g., NEXT_PUBLIC_API_BASE_URL_STAGING
    3. Hardcoded fallback (lowest priority)

Env Vars Simplified

From (mixed, confusing):

  • MODELSEED_API_URL, API_URL, etc.
  • REACT_APP_SOLR_COLLECTIONS_URL, manual Solr routing
  • NEXT_PUBLIC_USE_NEW_BIOCHEM (dead toggle)

To (canonical, clear):

  • NEXT_PUBLIC_DEPLOYMENT_MODE (staging/production/manual)
  • NEXT_PUBLIC_API_BASE_URL, NEXT_PUBLIC_REST_BASE_URL, NEXT_PUBLIC_STATUS_API_URL, NEXT_PUBLIC_SOLR_BASE_URL
  • NEXT_PUBLIC_SOLR_REACTIONS_COLLECTION, NEXT_PUBLIC_SOLR_COMPOUNDS_COLLECTION
  • Mode defaults: *_STAGING / *_PRODUCTION suffixes
  • No biochem backend toggle (Solr-backed by design)

Docker & Deployment

  • All canonical env vars wired as Docker build ARG and runtime env
  • deploy_container.sh auto-detects mode from git branch:
    • main/master/production → production mode
    • all others → staging mode
  • docker-compose.yml includes all env vars with mode-specific defaults

Pages Updated

  • Version/Status Page (/about/version): Shows deployment mode, checks services
  • Reactions Page (/biochemistry/reactions): Uses reactions_staging/reactions collection
  • Compounds Page (/biochemistry/compounds): Uses compounds_staging/compounds collection
  • All API Pages: Route through configured endpoints with feature-gate flags

Testing & Validation

  • New config.test.ts: 6 tests covering mode resolution, precedence, overrides, unset errors
  • Updated biochem.test.ts: 8 tests with new Solr collection naming
  • New reactionComments.test.ts: 2 tests for comments API route
  • All 90 tests pass
  • Build succeeds (Next.js compilation)
  • External endpoints verified: Solr cores, API services all HTTP 200

Documentation

  • lib/api/README.md: Config contract and precedence model
  • docs/TESTING.md: Staging/production testing guide
  • .env.example: Clear structure with all canonical vars and mode-specific defaults

Benefits

  • Simple mode switching: Just set NEXT_PUBLIC_DEPLOYMENT_MODE=staging|production
  • Docker Compose ready: All env vars properly wired
  • Backward compatible: Old env vars still resolve via fallback (no breaking changes)
  • Solr flexibility: Easy collection switching by mode or explicit override
  • Feature flags: Controlled by USE_MODELSEED_API, USE_NEW_PROXY
  • Strict unset mode: Errors early if mode is unset and overrides missing
  • Tested: Config resolution verified, all endpoints checked

Testing Instructions

# Local dev with staging defaults + localhost tunnel override
npm run dev  # requires SSH tunnel to port 8000

# Tests
npm run test:run  # all 90 tests pass

# Build
npm run build  # compiles successfully

# Docker
docker-compose build
docker-compose up

Notes

  • Backward compatibility: old env vars still work via hardcoded fallback
  • Next.js public env vars baked into build → rebuild needed for env changes
  • Deployment mode auto-detected from git branch in deploy_container.sh
  • Comments endpoint (/api/v0/comments) currently 404 on both staging/prod (backend issue, not env-related)

Checklist

  • Config resolution verified (6/6 tests)
  • Build compiles successfully
  • All 90 tests pass
  • External endpoints verified
  • Docker wiring complete
  • Pages updated
  • Documentation updated
  • Ready for staging/production split

Commits (7 atomic):

  1. refactor: implement canonical env contract with deployment mode
  2. feat: wire canonical env vars to Docker and deploy scripts
  3. refactor: update pages to use new config system
  4. feat: add reaction comments API route
  5. test: add comprehensive config and biochem tests
  6. docs: update scripts and documentation for new env system
  7. docs: update planning documents for env refactor completion

VibhavSetlur and others added 8 commits May 11, 2026 14:01
- Replace NEXT_PUBLIC_ENV_MODE with NEXT_PUBLIC_DEPLOYMENT_MODE
- Implement 3-level precedence: override → mode-default → fallback
- Remove dead NEXT_PUBLIC_USE_NEW_BIOCHEM toggle
- Add strict unset-mode validation
- Canonical names: site/API/REST/status/Solr endpoints
- Mode-specific defaults: _STAGING/_PRODUCTION suffixes
- Solr collections now required env vars (no app-level defaults)
- Biochem API stays Solr-backed by design
- Config exports: DEPLOYMENT_MODE, MODELSEED_API_URL, SOLR_BASE,
  SOLR_REACTIONS_COLLECTION, SOLR_COMPOUNDS_COLLECTION, etc.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add all canonical env vars as Docker build ARG
- Pass env vars to Next.js build and runtime
- Update docker-compose.yml with mode-specific defaults
- Implement auto-detection of deployment mode in deploy_container.sh
- git branch main/master/production → production mode
- All other branches → staging mode
- Ready for docker-compose and container deployments

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Version page: Use DEPLOY_ENV_LABEL for environment display
- StatusTable: Use MODELSEED_API_TEST_URL, SOLR_BASE, WORKSPACE_URL
- All pages: Reference config exports instead of env vars
- Reactions/Compounds pages: Route through Solr collections
- Feature-gated logic: Controlled by USE_MODELSEED_API, USE_NEW_PROXY
- Removed old env var references

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- New route: app/api/biochem/comments/route.ts
- Comments utility: lib/api/reactionComments.ts
- Modal UI updated to use new local route
- Separates comment handling from external REST API

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- New config.test.ts: 6 tests covering mode resolution, precedence, overrides
- New reactionComments.test.ts: 2 tests for comments API
- Updated biochem.test.ts: 8 tests with new Solr collections
- Updated setup.ts: Use DEPLOYMENT_MODE instead of ENV_MODE
- All tests pass (90/90)
- Config precedence fully validated

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- api-test.mjs: Use canonical env vars, normalized Solr URL handling
- All test scripts: Updated to use new config exports
- lib/api/README.md: Document config contract and precedence model
- tests/README.md: Updated for new env system
- docs/TESTING.md: Staging/production testing guide
- docs/TROUBLESHOOTING.md: Env var troubleshooting

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Phase 4: Comprehensive env system refactor and validation complete
- Verification: All endpoints tested, all tests pass, build successful
- Config resolution verified against staging and production services

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- 7 atomic commits implementing canonical env system
- Staging/production mode switching complete
- Solr collection routing working
- All tests passing (90/90)
- Ready for upstream PR

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 11, 2026 19:02
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the UI’s environment-variable contract to support canonical staging/production switching via NEXT_PUBLIC_DEPLOYMENT_MODE, updates API/Solr endpoint resolution accordingly, and adds a new reaction-comments client + Next.js proxy route.

Changes:

  • Introduces a centralized, mode-aware endpoint resolver in lib/api/config.ts (override → mode defaults → fallback) and updates Solr routing to use configurable core names.
  • Updates tests, scripts, Docker wiring, and documentation to use the new canonical env vars (NEXT_PUBLIC_API_BASE_URL, NEXT_PUBLIC_SOLR_*, etc.).
  • Adds reaction comment submission flow: UI modal now calls a typed client (lib/api/reactionComments.ts) which posts to a new Next.js route (app/api/biochem/comments).

Reviewed changes

Copilot reviewed 41 out of 41 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/utils/testHelpers.ts Switches test helper API base env var to the new canonical name.
tests/unit/utils/gridFiltering.test.ts Adjusts import strategy to allow env-dependent config initialization during tests.
tests/unit/api/reactionComments.test.ts Adds unit tests for the new reaction comments API client.
tests/unit/api/modelseed-client.test.ts Updates tests to use NEXT_PUBLIC_API_BASE_URL.
tests/unit/api/config.test.ts Adds coverage for deployment-mode precedence and required overrides in manual mode.
tests/unit/api/biochem.test.ts Updates biochem tests for mode-aware Solr collection routing and dynamic imports.
tests/unit/api/biochem-rest-filtering.test.ts Updates REST-mode biochem tests to use the new API base env var.
tests/setup.ts Establishes deterministic default deployment mode + Solr core names for tests.
tests/README.md Updates documented env var names for running tests.
tests/e2e/README.md Updates documented env var names for e2e setup.
scripts/test-reconstruct.mjs Renames API base env var usage to canonical NEXT_PUBLIC_API_BASE_URL.
scripts/test-reconstruct-upload.mjs Renames API base env var usage to canonical NEXT_PUBLIC_API_BASE_URL.
scripts/test-model-copy.mjs Renames API base env var usage to canonical NEXT_PUBLIC_API_BASE_URL.
scripts/test-fasta-upload.mjs Renames API base env var usage to canonical NEXT_PUBLIC_API_BASE_URL.
scripts/test-api-simple.mjs Renames API base env var usage to canonical NEXT_PUBLIC_API_BASE_URL.
scripts/diagnose-api.mjs Renames API base env var usage to canonical NEXT_PUBLIC_API_BASE_URL.
scripts/comprehensive-api-test.mjs Renames API base env var usage to canonical NEXT_PUBLIC_API_BASE_URL.
scripts/check-jobs.mjs Renames API base env var usage to canonical NEXT_PUBLIC_API_BASE_URL.
scripts/check_issues.mjs Renames API base env var usage to canonical NEXT_PUBLIC_API_BASE_URL.
scripts/api-test.mjs Implements deployment-mode-aware endpoint and Solr-core selection for the CLI API test suite.
README.md Updates top-level docs to reflect new deployment mode + canonical env vars.
lib/api/README.md Updates API docs to reflect Solr-backed biochem routing + new env contract.
lib/api/reactionComments.ts Adds client helper for submitting reaction comments via Next.js proxy.
lib/api/config.ts Implements canonical deployment-mode env resolution and Solr core routing constants.
lib/api/biochem.ts Updates Solr URL builder and ID lookups to use configured Solr collections.
docs/TROUBLESHOOTING.md Updates troubleshooting steps to use new env var names.
docs/TESTING.md Updates CI/testing env var references to new canonical names.
docs/TESTING-v3.0.0.md Updates versioned testing docs to new env var names.
Dockerfile Adds build ARGs for the expanded canonical env set.
docker-compose.yml Wires canonical env vars into build args and runtime environment.
deploy_container.sh Auto-detects deployment mode from git branch if not explicitly set.
components/ui/ReactionCommentModal.tsx Hooks modal submission into the new reaction comments client.
app/projects/page.tsx Switches an external image URL to HTTPS.
app/api/biochem/comments/route.ts Adds Next.js proxy route for submitting comments to legacy REST /comments.
app/about/version/StatusTable.tsx Uses new config constants for API/Solr status checks.
app/about/version/page.tsx Displays resolved environment label on the version page.
app/(user-data)/myMedia/page.tsx Updates error message to reference NEXT_PUBLIC_API_BASE_URL.
app/(user-data)/my-models/page.tsx Updates error message to reference NEXT_PUBLIC_API_BASE_URL.
.gsd/milestones/v1-alpha/3/VERIFICATION.md Updates milestone verification reference URL to HTTPS.
.gsd/milestones/v1-alpha/3/4-PLAN.md Updates plan document URL to HTTPS.
.env.example Rewrites the example env file to document canonical mode + endpoint precedence.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/api/config.ts Outdated
Comment on lines +8 to +10
function readEnv(name: string): string | undefined {
if (typeof process === 'undefined') return undefined;
return process.env[name];
Comment thread lib/api/config.ts
Comment on lines +253 to +254
toNonEmpty(readEnv('NEXT_PUBLIC_PROBMODELSEED_URL'))
?? `${MODELSEED_SITE_BASE_URL}/api/model`,
Comment thread app/api/biochem/comments/route.ts Outdated
Comment on lines +14 to +20
const body = (await request.json()) as CommentRequestBody;
const reactionId = typeof body.reactionId === 'string' ? body.reactionId.trim() : '';
const remarks = typeof body.remarks === 'string' ? body.remarks.trim() : '';
const email = typeof body.email === 'string' ? body.email.trim() : '';
const username = typeof body.username === 'string' ? body.username.trim() : '';
const isAlias = Boolean(body.isAlias);
const wrongStoichiometry = Boolean(body.wrongStoichiometry);
Comment on lines +6 to 9
it('supports OR logic for column filters', async () => {
process.env.NEXT_PUBLIC_DEPLOYMENT_MODE = 'staging';
const { filterDocsByGridModel } = await import('@/lib/api/biochem');
const rows = [
VibhavSetlur and others added 4 commits May 11, 2026 14:11
- Default NEXT_PUBLIC_DEPLOYMENT_MODE to staging when unset
- Add explicit manual mode for strict override validation
- Fix CI build failure in /api/biochem/comments route collection phase
- Restore profile-based docker compose staging/production services
- Update deploy_container.sh for safe mode selection and profile builds
- Correct production host-to-container port mapping (3001:3000)
- Update docs and config tests for manual/unset semantics

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Resolve deploy_container.sh conflict by combining profile selection with
  branch-based defaults and non-interactive safety
- Resolve docker-compose conflict with canonical env contract across staged
  and production profiles
- Keep production container mapping on host 3001 to container 3000

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Bump next to ^16.2.6 to clear high-severity advisories
- Align eslint-config-next to ^16.2.6
- Update lockfile and verify npm audit --omit=dev --audit-level=high passes
- Keep existing app behavior and build outputs unchanged

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix readEnv() to use static env access for proper Next.js inlining
- Add readEnvSafe() for dynamic key lookups while maintaining static map
- Add NEXT_PUBLIC_PROBMODELSEED_URL to env docs and Docker files
- Fix JSON parse error handling in comments route (return 400 on invalid)
- Fix boolean parsing in comments route (use typeof check)
- Fix test env pollution in gridFiltering.test.ts (use vi.stubEnv)

Verified: lint ✅, tests ✅, build ✅, docker compose ✅
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

- Comment out RDKIT in example (has good default in code)
- Verify all NEXT_PUBLIC_ vars from config.ts are in .env.example
- Total env vars: 33 NEXT_PUBLIC_ vars + 3 PATRIC_ vars
- Remove RDKit env var (has good default in code)
- Keep only PATRIC_TOKEN (remove username/password)
- Deployment mode: unset=staging, staging, production, manual all verified
- Unset NEXT_PUBLIC_DEPLOYMENT_MODE now = manual (strict) mode
- Requires override vars or explicit staging/production setting
- Updated test to match new behavior
- Unset NEXT_PUBLIC_DEPLOYMENT_MODE now defaults to staging (build-friendly)
- Manual mode still requires explicit overrides
- Test updated to match behavior
@VibhavSetlur VibhavSetlur merged commit a6def34 into ModelSEED:staging May 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants