Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .cursor/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@

- Before marking any task complete: `pnpm typecheck && pnpm lint && pnpm run -s format:check && pnpm test` must pass; coverage guard stays green.
- Update docs (plan, QA matrix, behaviour specs) as part of completion.
- Update docs (plan, QA matrix, behaviour specs) as part of completion.
- If a PR changes behaviour but does not update `docs/system_principles.md` and relevant guides/tests, CI should fail.

3. Use the Questions Log

Expand All @@ -47,7 +49,7 @@
7. Tests & docs as deliverables

- Add unit/integration tests for new logic and update `docs/qa/README.md` test mapping.
- Keep `docs/lm_behavior.md` and `docs/implementation.md` in sync with behaviour changes.
- Keep `docs/guide/reference/lm-behavior.md` and `docs/implementation.md` in sync with behaviour changes.

8. Communication discipline

Expand Down Expand Up @@ -98,7 +100,7 @@

16. Docs & Questions discipline

- Update `docs/implementation.md`, `docs/lm_behavior.md`, and `docs/qa/README.md` for any behaviour change.
- Update `docs/implementation.md`, `docs/guide/reference/lm-behavior.md`, and `docs/qa/README.md` for any behaviour change.
- Log uncertainties in `docs/questions.md`; proceed on safe defaults; revisit once answered.

17. Observability & safety
Expand All @@ -114,5 +116,5 @@

- Plan and task order: `docs/implementation.md`
- QA matrix and CI gates: `docs/qa/README.md`
- LM policy/behaviour: `docs/lm_behavior.md`
- LM policy/behaviour: `docs/guide/reference/lm-behavior.md`
- Questions log: `docs/questions.md`
2 changes: 1 addition & 1 deletion .cursor/rules/doc_links.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Links to critical files by name for Cursor memory and quick reference.
- [Project Structure](docs/project_structure.md)
- [Glossary](context/glossary.md)
- [Project Overview](context/project_overview.md)
- [System Principles](context/system_principles.md)
- [System Principles](docs/system_principles.md)

## Key Directories
- [Core Logic](core/)
Expand Down
47 changes: 47 additions & 0 deletions .cursor/rules/principles.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
alwaysApply: true
---
<!--══════════════════════════════════════════════════════════
╔══════════════════════════════════════════════════════════════╗
║ ░ P R I N C I P L E S ( S U M M A R Y ) ░░░░░░░░░░░░░░ ║
║ ║
║ ║
║ ║
║ ║
║ ╌╌ P L A C E H O L D E R ╌╌ ║
║ ║
║ ║
║ ║
║ ║
╚══════════════════════════════════════════════════════════════╝
• WHAT ▸ Minimal always-on principles for Cursor context
• WHY ▸ Conserve tokens while guiding behaviour
• HOW ▸ Abbreviated bullets; see docs/system_principles.md
-->

# Principles Snapshot

Human Flow & Dignity
- Human-first agency: auto-apply within band; no accept gesture; no expansion.
- Flow & rhythm: micro-corrections; defer heavy work during bursts.
- Low cognitive load: no suggestion lists; subtle underline/highlight; debug opt-in.
- Accessibility: respect reduced motion; SR announces; keyboard-first.

Safety, Trust & Integrity
- Caret-safe, non-undoing: never edit at/after caret; band-only; no undo entries.
- Local-first privacy: prefer local; remote off unless opted in; degrade gracefully; no text persistence.
- Explainability: show reasons, tiers, and truncations; toggleable explainers.
- Fail-soft: LM errors → rules-only; single-flight + abort; drop stale.

Adaptive Intelligence & Execution
- Context-minimal: smallest window; allow control JSON; outputs sanitized & clamped.
- Single-flight orchestration: one active gen per band; abort on input.
- Device-tier progressive: detect capabilities → tune cadence/tokens.
- Testable/observable: gates must pass; logs for merges/aborts/tiers.

Collaboration & Delivery
- Plan order & Questions: execute tasks in plan order; capture clarifications in `docs/questions.md`.
- Green gates: typecheck/lint/format/test must pass before merge.

See `docs/system_principles.md` for behaviours and examples.

26 changes: 25 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,29 @@
// • WHAT ▸ Custom dictionary entries for cSpell
// • WHY ▸ Silence false positives for product keywords
// • HOW ▸ Editor-only; no runtime impact
"cSpell.words": ["mindtyper", "mindtype", "gramm"]
"cSpell.words": [
"mindtyper",
"mindtype",
"gramm",
"behaviour",
"skimmable",
"guillemets",
"QXXX",
"desaturation",
"CARETSAFE",
"precsson",
"mindtypr",
"tooll",
"emdashes",
"lstens",
"cooldown",
"Qwen",
"WCAG",
"autoplay",
"testids",
"Workerize",
"cbindgen",
"sandboxed",
"webgpu"
]
}
6 changes: 3 additions & 3 deletions config/defaultThresholds.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,14 @@
• WHY ▸ Harmonises behaviour across engines/UX
• HOW ▸ Imported by engines and UI helpers
*/
export const SHORT_PAUSE_MS = 500; // aligned with plan/docs
export const SHORT_PAUSE_MS = 300; // perceptual rhythm default per principles
export const LONG_PAUSE_MS = 2000; // aligned with plan/docs
export const MAX_SWEEP_WINDOW = 80; // chars behind caret

// Mutable runtime-configurable thresholds (with safe defaults)
let typingTickMs = 75; // 60–90 ms sweet spot
let minValidationWords = 3;
let maxValidationWords = 8;
let minValidationWords = 5;
let maxValidationWords = 5;

// Accessors to support live tuning (demo controls)
export function getTypingTickMs(): number {
Expand Down
96 changes: 76 additions & 20 deletions core/diffusionController.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@ import {
getMaxValidationWords,
} from '../config/defaultThresholds';
import { tidySweep } from '../engines/tidySweep';
import { replaceRange } from '../utils/diff';
import type { LMAdapter } from './lm/types';
import { renderValidationBand, renderHighlight } from '../ui/highlighter';
import { createLogger } from './logger';

export interface DiffusionState {
text: string;
Expand All @@ -37,7 +39,14 @@ export interface BandPolicy {
// Context7 docs: Intl.Segmenter provides granularity: 'word' for word-like segments
// The isWordLike property indicates segments that are actual words vs punctuation/spaces
export function createDiffusionController(policy?: BandPolicy, _lmAdapter?: LMAdapter) {
const seg = new Intl.Segmenter(undefined, { granularity: 'word' });
// Safari/older browsers: Intl.Segmenter may be missing or partial. Provide a fallback.
let seg: Intl.Segmenter | null = null;
try {
seg = new Intl.Segmenter(undefined, { granularity: 'word' });
} catch {
seg = null;
}
const log = createLogger('diffusion');

let state: DiffusionState = { text: '', caret: 0, frontier: 0 };
// Throttle rendering to avoid UI storms (esp. Safari). ~60fps ceiling.
Expand All @@ -50,6 +59,19 @@ export function createDiffusionController(policy?: BandPolicy, _lmAdapter?: LMAd
lastRenderMs = now;
const renderRange = policy ? policy.computeRenderRange(state) : bandRange();
renderValidationBand(renderRange);
// Emit selection snapshot for LM inspector/debug
try {
const { start, end } = renderRange;
const ctxBefore = state.text.slice(Math.max(0, start - 60), start);
const span = state.text.slice(start, end);
const ctxAfter = state.text.slice(end, Math.min(state.text.length, end + 60));
(globalThis as unknown as Record<string, unknown>).__mtLastLMSelection = {
band: renderRange,
span,
ctxBefore,
ctxAfter,
};
} catch {}
}
}

Expand All @@ -63,22 +85,38 @@ export function createDiffusionController(policy?: BandPolicy, _lmAdapter?: LMAd
state.text = text;
state.caret = caret;
clampFrontier();
log.debug('update', { caret, frontier: state.frontier, textLen: text.length });
maybeRender();
}

function iterateWordSegments(slice: string): Array<{ index: number; segment: string }> {
const out: Array<{ index: number; segment: string }> = [];
if (seg) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
for (const s of (seg as any).segment(slice)) {
if ((s as { isWordLike?: boolean }).isWordLike)
out.push({ index: s.index, segment: s.segment });
}
return out;
}
// Fallback: unicode word run matches
const re = /[\p{L}\p{N}_]+/gu;
let m: RegExpExecArray | null;
while ((m = re.exec(slice))) {
out.push({ index: m.index, segment: m[0] });
}
return out;
}

function bandRange(): { start: number; end: number } {
// Compute a range covering min..max words behind caret, starting at frontier
const slice = state.text.slice(state.frontier, state.caret);
const words: Array<{ start: number; end: number }> = [];
for (const s of seg.segment(slice)) {
// Cast necessary due to incomplete TypeScript DOM types for Intl.Segmenter
// isWordLike property exists but not in TS lib DOM types yet
if ((s as { isWordLike?: boolean }).isWordLike) {
const start = state.frontier + s.index;
const end = start + s.segment.length;
words.push({ start, end });
}
}
const words: Array<{ start: number; end: number }> = iterateWordSegments(slice).map(
(s) => ({
start: state.frontier + s.index,
end: state.frontier + s.index + s.segment.length,
}),
);
if (words.length === 0) return { start: state.frontier, end: state.caret };
const minWords = getMinValidationWords();
const maxWords = getMaxValidationWords();
Expand All @@ -90,13 +128,11 @@ export function createDiffusionController(policy?: BandPolicy, _lmAdapter?: LMAd
function nextWordRange(): { start: number; end: number } | null {
if (state.frontier >= state.caret) return null;
const slice = state.text.slice(state.frontier, state.caret);
for (const s of seg.segment(slice)) {
// Cast necessary due to incomplete TypeScript DOM types for Intl.Segmenter
if ((s as { isWordLike?: boolean }).isWordLike) {
const start = state.frontier + s.index;
const end = start + s.segment.length;
if (end <= state.caret) return { start, end };
}
const segments = iterateWordSegments(slice);
for (const s of segments) {
const start = state.frontier + s.index;
const end = start + s.segment.length;
if (end <= state.caret) return { start, end };
}
return null;
}
Expand All @@ -106,8 +142,28 @@ export function createDiffusionController(policy?: BandPolicy, _lmAdapter?: LMAd
if (!r) return;
const res = tidySweep({ text: state.text, caret: state.caret, hint: r });
if (res.diff) {
renderHighlight({ start: res.diff.start, end: res.diff.end });
state.frontier = Math.max(state.frontier, res.diff.end);
// Do not log user text per privacy policy
log.debug('diff', {
start: res.diff.start,
end: res.diff.end,
});
// Apply the diff to local state for consistency with host
try {
const updated = replaceRange(
state.text,
res.diff.start,
res.diff.end,
res.diff.text,
state.caret,
);
state.text = updated;
} catch {
// If safety check fails, skip applying but still advance to avoid stalls
log.warn('replaceRange failed (safety)', { caret: state.caret });
}
renderHighlight({ start: res.diff.start, end: res.diff.end, text: res.diff.text });
const newEnd = res.diff.start + res.diff.text.length;
state.frontier = Math.max(state.frontier, newEnd);
} else {
// Even without a replacement, consider the word validated this tick
state.frontier = Math.max(state.frontier, r.end);
Expand Down
40 changes: 32 additions & 8 deletions core/lm/policy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ export interface SpanAndPrompt {
prompt: string | null;
span: string | null;
maxNewTokens: number;
controlJson: string;
}

export function selectSpanAndPrompt(
Expand All @@ -44,27 +45,50 @@ export function selectSpanAndPrompt(
cfg: LMBehaviorConfig = defaultLMBehaviorConfig,
): SpanAndPrompt {
const band = computeSimpleBand(text, caret);
if (!band) return { band: null, prompt: null, span: null, maxNewTokens: 0 };
if (!band)
return { band: null, prompt: null, span: null, maxNewTokens: 0, controlJson: '{}' };
const span = text.slice(band.start, band.end);
if (span.length < cfg.minSpanChars)
return { band: null, prompt: null, span: null, maxNewTokens: 0 };
return { band: null, prompt: null, span: null, maxNewTokens: 0, controlJson: '{}' };
if (span.length > cfg.maxSpanChars)
return { band: null, prompt: null, span: null, maxNewTokens: 0 };
return { band: null, prompt: null, span: null, maxNewTokens: 0, controlJson: '{}' };
if (cfg.enforceWordBoundaryAtEnd && /\w$/.test(span)) {
return { band: null, prompt: null, span: null, maxNewTokens: 0 };
return { band: null, prompt: null, span: null, maxNewTokens: 0, controlJson: '{}' };
}
const ctxLeft = Math.max(0, band.start - cfg.contextLeftChars);
const ctxRight = Math.min(text.length, band.end + cfg.contextRightChars);
const ctxBefore = text.slice(ctxLeft, band.start);
const ctxAfter = text.slice(band.end, ctxRight);
const instruction =
'Correct ONLY the Span. Do not add explanations or extra words. Return just the corrected Span.';
const prompt = `${instruction}\nContext before: «${ctxBefore}»\nSpan: «${span}»\nContext after: «${ctxAfter}»`;
const instruction = [
'Correct ONLY the Span. Return the corrected Span text exactly.',
'- No explanations or extra words',
'- No quotes or labels',
'- Keep meaning and style; fix grammar, clarity, and punctuation',
'- Keep length close to the Span (do not expand beyond it)',
].join('\n');
const control = {
mode: 'grammar_only',
tone: { formality: 0.0, warmth: 0.0, directness: 0.0 },
caps: { maxRewriteChars: cfg.maxSpanChars },
safety: { noExternalKnowledge: true },
v: 1,
};
const controlJson = JSON.stringify(control, null, 2);
const prompt = `${instruction}\n\nCONTROL (JSON): «${controlJson}»\nContext before: «${ctxBefore}»\nSpan: «${span}»\nContext after: «${ctxAfter}»`;
const maxNewTokens = Math.min(
Math.ceil(span.length * cfg.maxTokensFactor) + 6,
cfg.maxTokensCap,
);
return { band, prompt, span, maxNewTokens };
// Expose components for UI debug
(globalThis as unknown as Record<string, unknown>).__mtLastLMSelection = {
band,
prompt,
span,
ctxBefore,
ctxAfter,
controlJson,
};
return { band, prompt, span, maxNewTokens, controlJson };
}

export function postProcessLMOutput(
Expand Down
Loading
Loading