Skip to content

Add NodeNorm web frontends: GitHub Pages and Python-based#4

Draft
gaurav wants to merge 45 commits intobasic-implementation-in-uvfrom
add-nodenorm-frontend
Draft

Add NodeNorm web frontends: GitHub Pages and Python-based#4
gaurav wants to merge 45 commits intobasic-implementation-in-uvfrom
add-nodenorm-frontend

Conversation

@gaurav
Copy link
Copy Markdown
Collaborator

@gaurav gaurav commented Feb 16, 2026

This PR adds a web interface to babel-explorer, exposing all four tools — NodeNorm, XRefs, IDs, and Test Concordance — through a browser UI, a JSON REST API, and CSV downloads. It also includes the full CLI implementation, core query engine, and comprehensive test suite that the web frontend builds on.

WIP. TODO:

  • Think about moving the Python library into a subdirectory (python/?) for better organization, or at the least excluding web/ from being packaged into a future PyPI package.

Why FastAPI?

We considered several options for the web framework:

  • Streamlit — Easy to prototype with, but its re-run-the-whole-script model doesn't map well to babel-explorer's architecture (shared BabelDownloader/BabelXRefs instances with LRU caches, lazy multi-GB file downloads). It also wouldn't give us a clean REST API for programmatic access.
  • Flask — A solid choice, but we'd need to bolt on separate API documentation (Swagger/OpenAPI) and pick additional libraries for request validation.
  • Django — Too heavyweight for a tool with no database models or user auth.
  • FastAPI was the best fit because:
    • It auto-generates Swagger docs at /docs, giving us a REST API with interactive documentation for free.
    • It runs synchronous route handlers in a threadpool automatically, which is exactly what we need — the core code uses synchronous requests and DuckDB, and some queries trigger multi-GB Parquet downloads that would block an async event loop.
    • Jinja2 templating is a first-class integration, so we get server-rendered HTML without a JS build system.
    • Combined with htmx, forms submit via AJAX and swap in HTML fragments without writing any frontend JavaScript framework code.

What's included

  • Web app (src/babel_explorer/web/) — FastAPI app factory, routes, Jinja2 templates with Bootstrap 5 + htmx
  • Four tool pages with forms that return results as HTML tables via htmx
  • JSON REST API (/api/nodenorm, /api/xrefs, /api/ids, /api/test-concord) with query parameter interface
  • CSV downloads for all tools (/api/*/csv)
  • NodeNorm instance dropdown populated from NodeNorm.URLs with a "Custom URL..." option
  • CLI command babel-explorer web with --host, --port, --reload options
  • Navbar with links to all tools, Swagger docs, and GitHub repo
  • 25 unit tests (tests/test_web.py) covering HTML pages, htmx partials, JSON API, CSV endpoints, and helper functions — all mocked, no network needed
  • Core modules: BabelDownloader, BabelXRefs, NodeNorm, Click CLI, and 80+ tests across test_downloader.py, test_babel_xrefs.py, test_nodenorm.py

How to test

uv sync --group dev
uv run babel-explorer web          # http://127.0.0.1:8000
uv run pytest tests/test_web.py -v # 25 tests, ~0.4s

gaurav and others added 7 commits February 16, 2026 17:22
Adds a web UI (FastAPI + Jinja2 + htmx + Bootstrap 5) exposing NodeNorm,
XRefs, IDs, and Test Concordance via browser forms, a JSON REST API, and
CSV downloads. Launched with `babel-explorer web`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The NodeNorm and Test Concordance pages now show a select dropdown
populated from NodeNorm.URLs (defaulting to NodeNorm Dev), with a
"Custom URL..." option that reveals a free-text input.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CLI args are forwarded via environment variables so the reloaded
subprocess picks up the same config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add web route table and NodeNorm dropdown details to CLAUDE.md. Add web
frontend section to README.md covering startup, REST API examples, and
CSV download endpoints.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gaurav and others added 21 commits March 26, 2026 00:53
DB-dependent tools (XRefs, IDs) stay on the FastAPI/htmx server
(Kubernetes); API-only tools (NodeNorm, Test Concordance) move to
static GitHub Pages with direct browser fetch calls. This avoids
proxying CORS-enabled APIs through the server for no benefit and
establishes a reusable pattern for translator_sdk web demos.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Initialize Astro project in web/ with Vue 3 integration
- Create config/translator-endpoints.json as single source of truth
  for NodeNorm and NameRes deployment URLs across all environments
- Update nodenorm.py to load URLs from shared config instead of
  hardcoding them
- Add node_modules/ and web/dist/ to .gitignore

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- BaseLayout.astro + Navbar.astro matching Python frontend's Bootstrap
  dark-navbar style
- Landing page with tool card grid
- TypeScript types for NodeNorm API responses
- CURIE link-out support using biolink-model prefix map (v4.3.7)
- CurieLink.vue shared component for linked CURIEs
- web/README.md documenting dual-frontend architecture
- Fix .gitignore lib/ rule to not exclude web/src/lib/

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- nodenorm-api.ts: fetch wrapper for NodeNorm get_normalized_nodes
- NodeNormApp.vue: root Vue island orchestrating form + results
- NodeNormForm.vue: textarea, instance dropdown, API option toggles
- NodeNormResults.vue: accordion container with column visibility
- CurieResultCard.vue: per-CURIE accordion card with adaptive detail
  (full table for ≤10 equiv IDs, prefix summary + expand for more)
- EquivalentIdTable.vue: striped table with togglable columns
- ColumnVisibility.vue: page-wide column show/hide controls
- nodenorm.astro: Astro page hosting the Vue island

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SummaryCard.vue displays normalization success count, shared biolink
types across all results, and per-type breakdown. Wired into
NodeNormResults above the accordion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Mode toggle in NodeNormForm: "Single Instance" / "Compare Instances"
- Compare mode shows checkboxes to select multiple NodeNorm deployments
- NodeNormApp fetches from all selected instances in parallel using
  Promise.allSettled
- ComparisonView renders side-by-side table: rows = CURIEs, columns =
  instances, with preferred ID, label, types, equiv count per cell
- Rows where instances disagree on preferred ID are highlighted

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- FUTURE.md documents deferred features: deep diff comparison, URL
  state persistence, CSV export, additional tools, testing, CI deploy
- Fix navbar and landing page links missing slash between base path
  and page name (e.g. /babel-explorernodenorm → /babel-explorer/nodenorm)
- Add trailing slash to base in astro.config.mjs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…itecture

- CLAUDE.md: Add Astro/Vue frontend section (#6), shared config (#7),
  Astro directory structure, updated data flow, dev commands, file locations
- README.md: Replace single "Web Frontend" section with "Web Frontends"
  covering both Astro and Python frontends; update architecture section
  to reference web/ (not docs/) and config/translator-endpoints.json
  (not config.js)
- web/README.md: Document current NodeNorm features, component
  architecture, and how to add new tools

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix TypeError: routes passed expand= to get_curie_xrefs() whose
  parameter is named recurse=; would have failed at runtime on any
  xrefs request with expand=True or expand=False
- Extract _get_nodenorm() helper to replace 5 identical inline ternaries
- Extract _extract_extra_field_names() to replace duplicated loop
- Fix N+1 in _to_labeled_xref(): call get_identifier() once per CURIE
  instead of twice (halves cache lookups per cross-reference)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add 50 tests (28 library, 22 component) using Vitest + @vue/test-utils +
happy-dom. Shared JSON fixtures in tests/fixtures/ capture real NodeNorm
responses for use by both TypeScript and Python test suites.

Tests surfaced a bug in EquivalentIdTable.vue where the NodeNorm API returns
`description` as a string on equivalent identifiers, not string[]. Fixed
formatList() and the NormalizedIdentifier type definition.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NodeNorm API clarifications:
- id.description is a plain string (the "best" description, same as descriptions[0])
- equivalent_identifiers[].description is a plain string, not string[]
- top-level descriptions is string[] (all descriptions collected from the clique)

Fix NormalizedIdentifier.description type (string, not string | string[]).
Add id.description to NormalizedNode.id type definition.
Simplify EquivalentIdTable.vue description cell to use a plain truncate() helper.

Add fixtures for conflation testing (MESH:D014867 conflated vs no-conflate
shows 206 vs 30 equiv IDs), NCBIGene:1756 (gene with taxa), NCIT:C34373,
and a batch multi-CURIE response. Expand nodenorm-api tests with executable
documentation of the API response shape and conflation behaviour.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- CLAUDE.md: note babel-explorer npm package name, Vitest test suite (60
  tests), shared tests/fixtures/ directory, co-located test paths
- web/README.md: add Testing section linking to web/tests/README.md
- web/tests/README.md: expand fixture catalogue (9 fixtures including
  conflation variants and batch), add NodeNorm API shape reference (description
  field types, conflation behaviour, multi-CURIE GET vs POST), add POST
  example for fixture regeneration

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Encodes CURIEs, target instances, and non-default API options in query params
- Auto-submits on page load when URL contains curie= params
- Stop button aborts in-flight requests (AbortController/AbortSignal)
- Share button copies current URL to clipboard with a 2s "Copied!" flash
- Mode inferred from target= count (1 → single, 2+ → compare)
- Default CURIEs pre-populated so Normalize works on first click
- url-state.ts: readQueryState + buildQueryUrl helpers with full test coverage
- 77 tests passing (17 new)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- CLAUDE.md: bump test count 60 → 77, add url-state.ts to lib structure
- web/README.md: add shareable URLs feature, bump test count, add url-state.ts to arch diagram
- web/tests/README.md: add url-state.test.ts to test org tree, expand lib test descriptions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove Single/Compare mode toggle; instance selection is always checkboxes
- Add custom URL entry (input + Add button) below the known instance list
- Results always use the comparison table layout (works for 1 or N instances)
- Table rows are now clickable/expandable: click to reveal per-instance
  CurieDetailPanel with descriptions, types, IC score, and equiv ID table
- Extract CurieDetailPanel.vue from CurieResultCard.vue body content;
  used in both accordion cards and expanded table rows
- Delete NodeNormResults.vue (functionality absorbed into NodeNormApp)
- NodeNormApp: lift ColumnVisibility + SummaryCard, remove mode branching;
  single instance shows one SummaryCard, multiple shows per-instance row
- 92 tests passing (15 new: CurieDetailPanel + ComparisonView expansion)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ResultsSummary shows three stat tiles:
- Normalized: X/Y CURIEs found by all instances; lists partial hits
  (found by some but not all) and not-found CURIEs inline
- Disagreements: count of CURIEs where instances return different preferred
  IDs; hidden for single-instance queries; green when 0, amber otherwise
- Types: biolink type badges with ×N frequency counts aggregated across
  all instances and CURIEs

Tiles are data-driven (one computed array each), so adding new stats
requires no template restructuring. SummaryCard.vue and its tests removed.

99 tests passing (14 new in ResultsSummary.test.ts)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- CLAUDE.md: update component list (add CurieDetailPanel, ResultsSummary;
  remove NodeNormResults, SummaryCard), bump test count, revise hosted
  tools description
- web/README.md: rewrite features section (unified checkboxes, expandable
  rows, stat tiles), update architecture diagram, bump test count to 99
- web/tests/README.md: update test tree (ResultsSummary, CurieDetailPanel
  added; SummaryCard removed), expand component test descriptions,
  update future improvements to reflect current form design

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gaurav gaurav changed the title Add web frontend Add NodeNorm frontend Apr 1, 2026
@gaurav gaurav changed the title Add NodeNorm frontend Add NodeNorm GitHub Pages frontend Apr 1, 2026
@gaurav gaurav changed the title Add NodeNorm GitHub Pages frontend Add NodeNorm web frontends: GitHub Pages and Python-based Apr 1, 2026
gaurav and others added 17 commits April 1, 2026 10:27
- babel_xrefs: dedup by key tuple instead of hashing LabeledCrossReference
  (list[str] fields made it unhashable, crashing label_curies=True non-recursive path)
- routes: isinstance(LabeledCrossReference) instead of hasattr("subj_label");
  intern NodeNorm instances in app.state.nodenorm_cache so lru_cache persists
  across requests with a custom nodenorm_url
- NodeNormApp: hasResults as computed(() => resultsByInstance.size > 0)
  so it resets automatically when results are cleared on a new query
- downloader: replace double os.path.exists in __init__ with makedirs+isdir;
  replace exists-then-remove TOCTOU with direct remove + FileNotFoundError catch

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…vanced options

- NodeNormApp: wrap form in a query card (card mb-4) and results in a
  results card with card-header, card-body, card-footer; move results
  heading and ColumnVisibility into card-header; add Download JSON button
  in card-footer that exports {queried_curies, instances, results} keyed
  by CURIE → instance name
- NodeNormForm: split API options into main (Conflate, Drug/Chemical
  Conflate, always visible) and advanced (Description, Individual Types,
  Include Taxa) under a <details> collapsible; auto-opens when any
  advanced option differs from its default

The two-card pattern (query card / results card) establishes the reuse
convention for future GitHub Pages tools.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The toggle only affected EquivalentIdTable (visible only when a row is
expanded), so from the user's perspective clicking the buttons appeared
to do nothing. Fix: gate the type badge block in ComparisonView's main
summary row with v-if="visibleColumns.has('type')" so toggling Biolink
Type immediately hides/shows the badges in the main results table.

Add 27 tests covering:
- ColumnVisibility: rendering for various Set states, emits, reactivity
  when prop is replaced with a new Set instance
- EquivalentIdTable: each column header appears/hides per visibleColumns,
  row count, reactivity via setProps
- ComparisonView: type badges present/absent based on visibleColumns,
  updates when prop changes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ResultsSummary: rewrite typeCounts to use type[0] per CURIE (most-specific
  type, de-duped by distinct CURIEs); add selectedTypes prop + toggle-type-filter
  / clear-type-filter emits; render type badges as clickable buttons with
  active/inactive styling and a "clear" link
- ComparisonView: add typeFilter prop; compute visibleCuries to hide rows not
  matching the active filter; show empty-state message when all rows are filtered
- NodeNormApp: add typeFilter ref; wire toggleTypeFilter/clearTypeFilter; reset
  filter on new query; pass typeFilter to ComparisonView and selectedTypes to
  ResultsSummary
- Tests: fix 2 ResultsSummary tests that relied on old ancestor-type semantics;
  add 9 new tests covering filter interaction, button styling, emits, and
  reactivity in both components

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Display unique taxa from equivalent identifiers in the accordion header,
  controlled by the existing Taxa toggle (now on by default)
- Rename "Show columns:" label to "Show:" since toggles now affect both
  the detail table and the compressed card view

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
NodeNormForm initializes its internal state at creation time, before
onMounted fires in the parent. Moving readQueryState() to synchronous
setup code ensures the form receives the correct initial values on its
first render rather than falling back to the placeholder defaults.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Builds the Astro app and pushes to the gh-pages branch on every push
to main. Uses force_orphan so each deploy is a clean slate with no
leftover files from previous deployments.

Pull request trigger is included temporarily for testing — remove before
merging.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rather than showing the full node.type hierarchy (Gene → BiologicalEntity →
NamedThing…), compute "direct types" from the unique type values of each
equivalent identifier. For non-conflated results this is a single type;
for conflated results (e.g. Gene+Protein) it shows 2–5 types in
first-appearance order.

All type occurrences are now rendered as links to the biolink model docs
(https://biolink.github.io/biolink-model/{Type}). The individual_types
toggle is removed from the Advanced options UI since the display depends
on it always being enabled.

ResultsSummary type filter now uses direct types, so conflated CURIEs
appear under all their type buckets.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ComparisonView was missed in the previous change: it still used
node.type[0] for the type filter and node.type.slice(0,2) for the
summary row badges. Now uses getDirectTypes() for both, consistent
with the rest of the UI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each instance panel in the expanded CURIE detail row now shows a small
↗ link next to the instance name. Clicking it opens the NodeNorm
get_normalized_nodes GET response for that specific CURIE and instance
in a new tab, using the same API options that were used for the query.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant