You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-up to #94 — resolving the first model gap (the model text) the way the follow-up comment points to: extend the suggestion engine (#131), not bake a mode picker into project creation.
Additionally driven by a hard requirement: we want to fully support importing PT9 interlinearizations, and PT9's core glossing workflow (Options 1 & 3) uses a major-language project as the model text. So a major-language Scripture project must be a first-class model source here — not deferred.
Design principles this follows
Simplify project creation — attaching a model text is deferred, never a creation-time commitment.
Generalize the interlinearizer — a project isn't locked to one kind of suggestion source. The model can be a sibling interlinear project or a major-language Scripture project, attached, changed, or dropped whenever the analyst wants.
First-contact discoverable — the model surfaces intuitively at the point suggestions matter, and isn't worried about before then.
Reframe
#94's original framing tied the model text to a 4-way "Create new" mode chosen at project creation (PT9 Options 1 & 3). This issue takes the opposite approach:
A model text is a detachable suggestion source, attached/removed at any time via the project metadata modal / project settings — never a creation-time decision.
The #131 engine reuses approved glosses keyed by normalized surface form within the current draft (suggestion-engine.ts, selectPoolIndex). A model source extends that pool. Two kinds, matched two different ways:
Sibling interlinear project — surface-form reuse. Its approved TokenAnalysis glosses merge into the pool and match the source by surface form. Produces hits when it shares vocabulary with the source (same source text, a prior edition, a shared "master gloss" project). A direct extension of Add engine to generate token gloss suggestions from previous glosses #131. Also the natural landing spot for imported PT9 glosses (see below).
Major-language Scripture project — alignment-based (PT9's model text). A plain major-language translation (e.g. an English Bible) whose words are the gloss vocabulary. It is in a different language from the source, so surface-form matching does not apply. Suggestions come from source↔model word alignment: source word → aligned model word(s) → suggested gloss. Required for full PT9 interlinearization support (Options 1 & 3).
When a PT9 interlinearization is imported, its clusters land as approved TokenAnalysis glosses. Those immediately enter the surface-form pool, so repeat occurrences of an already-glossed source word get suggestions from Add engine to generate token gloss suggestions from previous glosses #131's existing mechanism — no alignment needed for the already-glossed vocabulary.
The major-language model project is then what supplies suggestions for source words the import never glossed, via alignment. So the two mechanisms compose: imported glosses cover what PT9 already did; the model covers the rest.
The end-to-end PT9 XML → InterlinearProjectimport pipeline (mapping clusters/lexemes/glosses to analyses, resolving gloss text, linking the model + output projects, an import command/UI) is broader than this issue — tracked in #150. This issue covers only the model-text/suggestion side.
Model additions
InterlinearProject.modelProjectId?: string — the Platform.Bible project used as the suggestion source. May reference either a sibling interlinear project (→ surface-form reuse) or a plain major-language Scripture project (→ alignment). The engine chooses the resolution strategy by whether an interlinear analysis exists for the referenced project.
DraftProject.modelProjectId?: string — mirror it on the draft (exactly as targetProjectId is mirrored) so the model is live while editing, before Save As persists it.
On Save As, copy modelProjectId from draft → new InterlinearProject. On Open, seed it from the project → draft.
Engine / store additions
Surface-form tier (sibling interlinear model): load the model's TextAnalysis once (via interlinearizer.getProject), build its pool with buildPoolIndex, and seed it as a staticmodelPoolIndex that does not recompute on the draft's own edits. deriveTokenSuggestion gains a model tier: the draft's approved bucket owns the suggested slot; model entries the draft lacks become additional candidates.
Alignment tier (major-language model): a source↔model word-alignment step maps each source token to model word(s), whose text becomes the suggested gloss. Alignment can be seeded from imported PT9 cluster→gloss data and/or computed (eflomal / fast_align), and could be persisted via AlignmentLink. This is the larger part of the work.
Model-derived payloads carry a distinct producer (e.g. "model:{id}") so SuggestionDropdown can badge "from model" and they're never confused with local suggestions.
AnalysisStoreProvider gains a modelPoolIndex (or modelAnalysis) prop, threaded from the loader.
Settings additions
Add interlinearizer.useModelSuggestions: boolean to ProjectSettingTypes — a per-project view toggle to include/suppress model suggestions (parallels the existing showSuggestions demo flag, and lets users mute the model without detaching it). The which-project pointer stays on the model (above); only the on/off toggle is a setting.
Key technical wrinkle
buildPoolIndex documents that keying on surface form alone (ignoring writing system) is "correct for v1: the pool is a single source project whose word tokens share one writing system." That assumption:
Holds for a sibling interlinear model only when its writing system matches the source; otherwise the key must extend to (writingSystem, surfaceForm).
Is moot for a major-language model — cross-language surface forms never match, which is exactly why that path uses alignment rather than the pool key.
Either way the model's gloss language must overlap the project's analysisLanguages for gloss?.[analysisLanguage] to resolve.
One field or two: a single modelProjectId for both model kinds (resolved by inspection), or separate modelInterlinearProjectId / modelTextProjectId?
Alignment approach for the major-language model: reuse imported PT9 cluster→gloss mappings, compute alignment (eflomal / fast_align), or both? Persist via AlignmentLink?
Explicit pointer vs. implicit shared pool — for the sibling-interlinear kind, should the pool implicitly widen to include all of the user's approved glosses on the same source instead of an explicit pointer? Explicit is recommended (matches PT9's mental model, gives control).
Should this go to user-questions.md for review outside the dev team, given it decides suggestion UX? (per AGENTS.md UX-decisions guidance)
Size: L — surface-form reuse alone is M; the alignment-based major-language model (required for PT9 parity) is the larger part. Priority: P2 for the suggestion enhancement; the major-language model path is a prerequisite for full PT9 import parity.
Follow-up to #94 — resolving the first model gap (the model text) the way the follow-up comment points to: extend the suggestion engine (#131), not bake a mode picker into project creation.
Additionally driven by a hard requirement: we want to fully support importing PT9 interlinearizations, and PT9's core glossing workflow (Options 1 & 3) uses a major-language project as the model text. So a major-language Scripture project must be a first-class model source here — not deferred.
Design principles this follows
Reframe
#94's original framing tied the model text to a 4-way "Create new" mode chosen at project creation (PT9 Options 1 & 3). This issue takes the opposite approach:
suggested/candidate, never auto-approved.interlinearModediscriminator (that output concern is its own issue — see the companion output issue Wire the output project (back-translation / adaptation) into export, not creation #149).Two kinds of model source (both in scope)
The #131 engine reuses approved glosses keyed by normalized surface form within the current draft (
suggestion-engine.ts,selectPoolIndex). A model source extends that pool. Two kinds, matched two different ways:TokenAnalysisglosses merge into the pool and match the source by surface form. Produces hits when it shares vocabulary with the source (same source text, a prior edition, a shared "master gloss" project). A direct extension of Add engine to generate token gloss suggestions from previous glosses #131. Also the natural landing spot for imported PT9 glosses (see below).PT9 import interplay
The PT9 XML parser already exists (
src/parsers/pt9/interlinearXmlParser.ts, schema) but is not wired into any import feature yet. Two things fall out for this issue:TokenAnalysisglosses. Those immediately enter the surface-form pool, so repeat occurrences of an already-glossed source word get suggestions from Add engine to generate token gloss suggestions from previous glosses #131's existing mechanism — no alignment needed for the already-glossed vocabulary.Model additions
InterlinearProject.modelProjectId?: string— the Platform.Bible project used as the suggestion source. May reference either a sibling interlinear project (→ surface-form reuse) or a plain major-language Scripture project (→ alignment). The engine chooses the resolution strategy by whether an interlinear analysis exists for the referenced project.DraftProject.modelProjectId?: string— mirror it on the draft (exactly astargetProjectIdis mirrored) so the model is live while editing, before Save As persists it.modelProjectIdfrom draft → newInterlinearProject. On Open, seed it from the project → draft.Engine / store additions
TextAnalysisonce (viainterlinearizer.getProject), build its pool withbuildPoolIndex, and seed it as a staticmodelPoolIndexthat does not recompute on the draft's own edits.deriveTokenSuggestiongains a model tier: the draft's approved bucket owns thesuggestedslot; model entries the draft lacks become additionalcandidates.fast_align), and could be persisted viaAlignmentLink. This is the larger part of the work.producer(e.g."model:{id}") soSuggestionDropdowncan badge "from model" and they're never confused with local suggestions.AnalysisStoreProvidergains amodelPoolIndex(ormodelAnalysis) prop, threaded from the loader.Settings additions
interlinearizer.useModelSuggestions: booleantoProjectSettingTypes— a per-project view toggle to include/suppress model suggestions (parallels the existingshowSuggestionsdemo flag, and lets users mute the model without detaching it). The which-project pointer stays on the model (above); only the on/off toggle is a setting.Key technical wrinkle
buildPoolIndexdocuments that keying on surface form alone (ignoring writing system) is "correct for v1: the pool is a single source project whose word tokens share one writing system." That assumption:(writingSystem, surfaceForm).Either way the model's gloss language must overlap the project's
analysisLanguagesforgloss?.[analysisLanguage]to resolve.Out of scope
interlinearMode('back-translation' | 'adaptation') — the Thoroughly evaluate the 4 PT9 interlinearizer choices #94 gap Set up CodeRabbitAI #2 output discriminator; tracked in the companion output issue Wire the output project (back-translation / adaptation) into export, not creation #149.Open questions
modelProjectIdfor both model kinds (resolved by inspection), or separatemodelInterlinearProjectId/modelTextProjectId?fast_align), or both? Persist viaAlignmentLink?user-questions.mdfor review outside the dev team, given it decides suggestion UX? (per AGENTS.md UX-decisions guidance)Size: L — surface-form reuse alone is M; the alignment-based major-language model (required for PT9 parity) is the larger part.
Priority: P2 for the suggestion enhancement; the major-language model path is a prerequisite for full PT9 import parity.