Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 32 additions & 21 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,24 +117,25 @@ cd web && npm run test:watch # Watch mode
- Astro + Vue 3 static site for browser-only tools (no backend required)
- npm package name: `babel-explorer` (directory stays `web/`)
- Deployed to GitHub Pages
- Currently hosts: NodeNorm lookup (unified instance selection via checkboxes, expandable result table, shareable URLs)
- Calls NodeNorm API directly from the browser via `fetch()` (CORS-enabled)
- Currently hosts:
- **NodeNorm lookup** — unified instance selection, expandable result table, shareable URLs
- **NameRes lookup** — batch name → CURIE resolution with expected-CURIE validation via `[[CURIE]]` annotations
- **Autocomplete playground** — live-as-you-type NameRes with preset dropdown (Disease/Gene/SmallMolecule/Custom), debounce+abort, latency badges, expected-CURIE rank check, and side-by-side environment comparison
- Calls NodeNorm and NameRes APIs directly from the browser via `fetch()` (CORS-enabled)
- Uses Bootstrap 5 (CDN) with the same dark-navbar styling as the Python frontend
- CURIE link-outs via [biolink-model prefix map](https://github.com/biolink/biolink-model) (v4.3.7, fetched at runtime)
- Tested with Vitest + @vue/test-utils + happy-dom (99 tests); see `web/tests/README.md`
- Tested with Vitest + @vue/test-utils + happy-dom (276 tests); see `web/tests/README.md`
- See `web/README.md` for development instructions and `web/FUTURE.md` for deferred features

7. **Shared Configuration** (`config/`):
- `config/translator-endpoints.json` — single source of truth for NodeNorm and NameRes deployment URLs across all environments (dev, exp, ci, test, prod)
- Consumed by Python CLI/frontend (`nodenorm.py`) and Astro frontend (`NodeNormApp.vue`)
- `config/translator-endpoints.json` — single source of truth for NodeNorm and NameRes deployment URLs across all environments
- Consumed by Python CLI/frontend (`nodenorm.py`) and Astro frontend (`NodeNormApp.vue`, `NameResApp.vue`, `AutocompleteApp.vue`)
- **NodeNorm envs**: `dev`, `exp`, `ci` (ElasticSearch-backed CI), `redis_ci` (Redis CI — kept for comparison, to be retired), `test`, `prod`
- **NameRes envs**: `dev`, `exp`, `ci` (Solr-backed CI — the stable default), `es_ci` (ElasticSearch CI — experimental), `test`, `prod`
- Note: NodeNorm and NameRes use the same `ci` key but point to different backends. NodeNorm CI is ES-based; NameRes CI is still Solr-based. The NameRes `es_ci` key is a separate experimental instance.

### Data Flow

1. User provides CURIEs via CLI
2. BabelDownloader ensures required Parquet files are downloaded
3. BabelXRefs queries files using DuckDB
4. If `--labels` or `--recurse` flags are set, NodeNorm is queried for additional metadata
5. Results are printed to stdout
1. User provides CURIEs via CLI, Python web UI, or Astro web UI
2. For DB-dependent tools (XRefs, IDs): BabelDownloader ensures required Parquet files are downloaded, BabelXRefs queries files using DuckDB
3. For API-only tools (NodeNorm): the Astro frontend calls the NodeNorm API directly from the browser; the Python frontend proxies through the server
Expand All @@ -148,25 +149,32 @@ web/src/
layouts/BaseLayout.astro # Bootstrap 5 CDN + dark navbar
components/
Navbar.astro # Dark navbar matching Python frontend
nodenorm/
NodeNormApp.vue # Root Vue island (client:only="vue")
NodeNormForm.vue # Form: textarea, checkboxes + custom URL, API toggles
ComparisonView.vue # Results table (rows=CURIEs, cols=instances) with expandable rows
CurieDetailPanel.vue # Expandable detail body: description, types, IC, equiv IDs
CurieResultCard.vue # Accordion card wrapping CurieDetailPanel (single-instance use)
ResultsSummary.vue # Unified stat tiles: normalized count, disagreements, types
EquivalentIdTable.vue # Striped table with togglable columns
ColumnVisibility.vue # Page-wide column show/hide controls
nodenorm/ # NodeNorm tool components (App, Form, ComparisonView, CurieDetailPanel, CurieResultCard, ResultsSummary, EquivalentIdTable, ColumnVisibility)
nameres/ # NameRes tool components (App, Form, ComparisonView, DetailPanel, ResultsSummary)
autocomplete/ # Autocomplete tool components (App, Form, Results, ComparisonView, HighlightedFragment, ExpectedCuriePanel, LatencyBadge)
shared/
InstanceSelector.vue # Reusable env checkboxes + custom URL + localStorage prefs
CurieLink.vue # CURIE → external URL link using biolink prefix map
BiolinkTypeLink.vue # Biolink type badge → link to biolink.github.io/biolink-model/{Type}; handles both prefixed ("biolink:Gene") and unprefixed ("Gene") types; used by NodeNorm, NameRes, and Autocomplete
lib/
nodenorm-api.ts # fetch() wrapper for NodeNorm get_normalized_nodes (AbortSignal support)
nameres-api.ts # fetch() wrapper for NameRes /lookup; parseSearchTerms; validateExpectedCuries
nameres-types.ts # NameResResult, NameResApiOptions, DEFAULT_NAMERES_OPTIONS
nameres-url-state.ts # Encode/decode NameRes query state
autocomplete-url-state.ts # Encode/decode Autocomplete query state (q, preset, target, expected, debounce, highlight)
autocomplete-presets.ts # Disease/Gene/SmallMolecule/Custom preset definitions + detectPreset
autocomplete-diff.ts # computeInstanceDiffs (presence/rank/label/types mismatch) + classifyExpectedCurie
debounce.ts # debounce(fn, ms) with .cancel(); synchronous path when ms===0
highlight-sanitize.ts # Whitelist sanitizer for Solr highlighting (<em> only)
instance-prefs.ts # sessionPrefs + localStorage helpers + sortInstances (ENV_ORDER canonical sort) — used by InstanceSelector and all App components
curie-links.ts # Biolink prefix map loader + URL builder
url-state.ts # Encode/decode query state in URL params (readQueryState, buildQueryUrl)
types.ts # TypeScript interfaces + DEFAULT_API_OPTIONS
url-state.ts # Encode/decode NodeNorm query state
types.ts # NodeNorm TypeScript interfaces + DEFAULT_API_OPTIONS
pages/
index.astro # Landing page with tool cards
nodenorm.astro # NodeNorm tool page
nameres.astro # NameRes tool page
autocomplete.astro # Autocomplete playground page
```

Key patterns:
Expand Down Expand Up @@ -195,6 +203,8 @@ The NodeNorm and Test Concordance pages include a dropdown to select from predef
- **DuckDB for querying**: In-memory SQL queries against Parquet files for fast lookups
- **Dual frontend**: DB-dependent tools on a server (FastAPI + htmx), API-only tools in the browser (Astro + Vue). Split follows data dependencies.
- **Shared config**: `config/translator-endpoints.json` is the single source of truth for deployment URLs, consumed by both frontends and the CLI
- **Canonical instance ordering**: `sortInstances()` in `instance-prefs.ts` sorts by `ENV_ORDER` (Exp → Dev → CI → ES CI → Redis CI → Test → Prod → custom URLs). Called in every App component after setting `queriedInstances` so comparison tables always display in pipeline order.
- **Adding a new named instance**: requires (1) entry in `config/translator-endpoints.json`, (2) add env key to `PRIMARY_ENVS` in `InstanceSelector.vue` if it should appear in the main section rather than "Extended environments", (3) add to `ENV_ORDER` in `instance-prefs.ts` for sort position, (4) add label to `ENV_LABELS` in the relevant App component(s) and in `InstanceSelector.vue`, (5) add a `labelHtml()` case in `InstanceSelector.vue` if the label needs `<abbr>` rendering (e.g. ES, Redis), (6) add to Python `env_labels` in `nodenorm.py` if it's a NodeNorm instance.

## Testing

Expand Down Expand Up @@ -231,6 +241,7 @@ Tests live in `tests/` and are split into fast **unit tests** (mocked, no networ

- **Data directory**: The `data/` directory is gitignored and contains downloaded Parquet files and generated DuckDB databases
- **Babel versions**: The default Babel version is `2025nov19`, but this can be customized via `--local-dir` and `--babel-url`
- **Pre-existing Python test failures**: `tests/test_web.py::test_nodenorm_page` and `tests/test_web.py::test_test_concord_page` fail with `AttributeError: type object 'NodeNorm' has no attribute 'URLs'` — this is a pre-existing issue unrelated to the Astro frontend work. The `NodeNorm.URLs` attribute referenced in the Python web routes no longer matches the actual class interface.

## File Locations

Expand Down
12 changes: 7 additions & 5 deletions config/translator-endpoints.json
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
{
"nodenorm": {
"dev": "https://nodenormalization-sri.renci.org/",
"exp": "https://nodenormalization-exp.apps.renci.org/",
"ci": "https://nodenorm.ci.transltr.io/",
"test": "https://nodenorm.test.transltr.io/",
"prod": "https://nodenorm.transltr.io/"
"dev": "https://nodenormalization-sri.renci.org/",
"exp": "https://nodenormalization-exp.apps.renci.org/",
"ci": "https://nodenorm-es.ci.transltr.io/",
"redis_ci": "https://nodenorm.ci.transltr.io/",
"test": "https://nodenorm.test.transltr.io/",
"prod": "https://nodenorm.transltr.io/"
},
"nameres": {
"dev": "https://name-resolution-sri.renci.org/",
"exp": "https://name-resolution-exp.apps.renci.org/",
"ci": "https://name-lookup.ci.transltr.io/",
"es_ci": "https://namelookup-es.ci.transltr.io/",
"test": "https://name-lookup.test.transltr.io/",
"prod": "https://name-lookup.transltr.io/"
}
Expand Down
11 changes: 6 additions & 5 deletions src/babel_explorer/core/nodenorm.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,12 @@ def _load_nodenorm_urls() -> dict[str, str]:
try:
endpoints = json.loads(config_path.read_text())
env_labels = {
"dev": "Dev",
"exp": "Exp",
"ci": "CI",
"test": "Test",
"prod": "Production",
"dev": "Dev",
"exp": "Exp",
"ci": "CI",
"redis_ci": "Redis CI",
"test": "Test",
"prod": "Production",
}
return {
f"NodeNorm {env_labels.get(env, env)}": url
Expand Down
56 changes: 42 additions & 14 deletions web/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ babel-explorer has **two** web frontends:

| Frontend | Stack | Deployment | Tools | Directory |
|----------|-------|------------|-------|-----------|
| **This one** | Astro + Vue 3 | GitHub Pages | NodeNorm (more planned) | `web/` |
| **This one** | Astro + Vue 3 | GitHub Pages | NodeNorm, NameRes, Autocomplete | `web/` |
| **Python frontend** | FastAPI + htmx | Kubernetes | XRefs, IDs, Test Concordance, NodeNorm | `src/babel_explorer/web/` |

The split follows data dependencies: tools that only call external APIs (NodeNorm, NameRes) can run entirely in the browser. Tools that query multi-GB Parquet files via DuckDB need a server.
Expand All @@ -20,14 +20,35 @@ Both frontends share the same Bootstrap 5 dark-navbar styling for visual consist
### NodeNorm Lookup (`/nodenorm`)

- **Bulk normalization**: Enter multiple CURIEs, toggle API options (conflation, descriptions, individual types, taxa)
- **Unified instance selection**: Checkboxes for known NodeNorm deployments (Dev, Exp, CI, Test, Prod) plus a custom URL input; any combination of instances can be queried together
- **Unified instance selection**: Checkboxes for known NodeNorm deployments (Dev, Exp, ES CI, Redis CI, Test, Prod) plus a custom URL input; any combination of instances can be queried together
- **Comparison table**: Results shown as a table — rows = CURIEs, columns = selected instances; rows highlighted amber when instances disagree on preferred ID
- **Expandable row detail**: Click any CURIE row to reveal per-instance panels showing description, biolink types, IC score, and equivalent identifiers (prefix summary + expand/collapse for large cliques)
- **Column visibility**: Toggle biolink type, taxa, description columns page-wide
- **Unified summary**: Stat tiles above the table — normalized count (with partial/not-found detail), disagreement count across instances, and biolink type frequency badges
- **Shareable URLs**: Query state encoded in URL params (`?curie=`, `?target=`, non-default options); Share button copies link to clipboard; auto-submits on page load when URL contains CURIEs
- **CURIE link-outs**: Identifiers link to external resources via [biolink-model prefix map](https://github.com/biolink/biolink-model) (v4.3.7)

### NameRes Lookup (`/nameres`)

- **Batch name → CURIE resolution**: Enter multiple search terms (one per line), see ranked results per instance
- **Expected-CURIE validation**: Annotate any line with `[[CURIE]]` to mark expected results; validation reports success/partial/failure with the rank of the best match, configurable via a "fail if in top N" threshold
- **Multi-instance comparison**: Same unified instance selector as NodeNorm
- **API tuning**: `biolink_type`, `only_prefixes`, `exclude_prefixes`, `only_taxa` exposed in the form; limit and autocomplete mode toggles; shareable URL state and JSON export

### Autocomplete Playground (`/autocomplete`)

Purpose-built for evaluating NameRes as a real autocomplete (the primary way the Translator UI uses it).

- **Live-as-you-type**: every keystroke re-queries after a configurable debounce (0/150/300/500 ms); in-flight requests are aborted via `AbortController` so stale responses never render
- **Preset dropdown**: one-click switches between the three Translator UI query shapes — Disease (`DiseaseOrPhenotypicFeature` + `only_prefixes=MONDO|HP`), Gene, Small Molecule — plus Custom. Presets only set `biolink_type` and `only_prefixes`; every other field stays user-editable
- **Advanced options**: debounce, autocomplete flag, highlighting toggle, exclude_prefixes, only_taxa (collapsed by default, auto-opens when any non-default is set)
- **Latency badges**: per-instance response time, with a tooltip noting parallel-contention when comparing multiple environments
- **Match-reason highlighting**: renders NameRes's Solr `highlighting` fragments (`<em>`-wrapped matches) behind a whitelist sanitiser — the only place `v-html` is used in the codebase
- **Single- vs multi-instance views**: one instance → rich ranked table with copy-API-URL per row; multiple → side-by-side comparison table with row/cell styling for missing CURIEs, rank drift, label/types mismatches; each column header has an `API↗` link to the raw NameRes JSON for that instance
- **Biolink type display**: the left-hand CURIE column shows the most specific Biolink type as a badge (linked to biolink.github.io/biolink-model); when instances disagree, all distinct types are shown and the type is repeated per-cell for easy comparison
- **Expected-CURIE panel**: paste CURIEs you expect to see; "Check" button fires a `limit=100` parallel lookup per instance and shows whether each expected CURIE appears in the top-N (green), top-100 (amber) or is missing (red). Round-trips through URL state for shareable review links
- **Shareable state**: `q`, `preset`, repeated `target`, repeated `expected`, per-field option overrides, non-default `debounce` and `highlight` all encoded in the URL

## Development

```bash
Expand All @@ -41,7 +62,7 @@ This starts a local dev server at `http://localhost:4321/babel-explorer/`.
## Testing

```bash
npm test # Run all 99 Vitest unit + component tests
npm test # Run all Vitest unit + component tests (~276 tests across lib and components)
npm run test:watch # Watch mode
```

Expand Down Expand Up @@ -78,24 +99,31 @@ src/
pages/
index.astro # Landing page with tool cards
nodenorm.astro # Hosts NodeNormApp Vue island
nameres.astro # Hosts NameResApp Vue island
autocomplete.astro # Hosts AutocompleteApp Vue island
components/
Navbar.astro # Shared navbar (Astro component)
nodenorm/ # NodeNorm Vue components
NodeNormApp.vue # Root island: orchestrates form + results
NodeNormForm.vue # CURIE input, checkbox instance selection, custom URL, API options
ComparisonView.vue # Results table with expandable per-CURIE rows
CurieDetailPanel.vue # Detail body: description, types, IC, equiv IDs table
CurieResultCard.vue # Accordion card wrapping CurieDetailPanel
ResultsSummary.vue # Stat tiles: normalized count, disagreements, type badges
EquivalentIdTable.vue # Equiv ID table with togglable columns
ColumnVisibility.vue # Column show/hide controls
nodenorm/ # NodeNorm Vue components (App, Form, ComparisonView, CurieDetailPanel, CurieResultCard, ResultsSummary, EquivalentIdTable, ColumnVisibility)
nameres/ # NameRes Vue components (App, Form, ComparisonView, DetailPanel, ResultsSummary)
autocomplete/ # Autocomplete Vue components (App, Form, Results, ComparisonView, HighlightedFragment, ExpectedCuriePanel, LatencyBadge)
shared/
InstanceSelector.vue # Env checkboxes + custom URL + localStorage prefs (used by all tools)
CurieLink.vue # CURIE → external URL link
BiolinkTypeLink.vue # Biolink type → link to biolink.github.io/biolink-model/{Type}; accepts both "biolink:Gene" and "Gene"
lib/
nodenorm-api.ts # NodeNorm API fetch wrapper (supports AbortSignal)
nameres-api.ts # NameRes /lookup wrapper; parseSearchTerms; validateExpectedCuries
nameres-types.ts # NameRes TypeScript interfaces + DEFAULT_NAMERES_OPTIONS
nameres-url-state.ts # NameRes URL-state encode/decode
autocomplete-url-state.ts # Autocomplete URL-state encode/decode + DEFAULT_AUTOCOMPLETE_OPTIONS + parseExpectedCuries
autocomplete-presets.ts # Disease/Gene/SmallMolecule/Custom presets + detectPreset
autocomplete-diff.ts # Cross-instance diff signals + expected-CURIE classification
debounce.ts # debounce(fn, ms) with .cancel() and synchronous 0-ms path
highlight-sanitize.ts # Whitelist sanitizer — allow only <em>/</em>
instance-prefs.ts # sessionPrefs + localStorage helpers + sortInstances (ENV_ORDER canonical sort)
curie-links.ts # Biolink prefix map loader
url-state.ts # Encode/decode query state in URL params
types.ts # TypeScript interfaces
url-state.ts # NodeNorm URL-state encode/decode
types.ts # NodeNorm TypeScript interfaces
```

## Adding a New Tool
Expand Down
5 changes: 5 additions & 0 deletions web/src/components/Navbar.astro
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ const base = rawBase.endsWith('/') ? rawBase : rawBase + '/';
NameRes
</a>
</li>
<li class="nav-item">
<a class:list={['nav-link', { active: activePage === 'autocomplete' }]} href={`${base}autocomplete`}>
Autocomplete
</a>
</li>
</ul>
<ul class="navbar-nav ms-auto">
<li class="nav-item">
Expand Down
Loading
Loading