feat: reusable MaskEditor UI component with browser-ML assists

[shaoster]

Vision: Pottery Computer Vision Pipeline

Problem / Motivation

Every issue in Milestone #2 that produces training labels — in-app annotation (#420), the "Report bad crop" correction flow (#421), and the A/B disagreement re-labeling queue (#418) — needs the same core surface: a component that takes a source image and an optional candidate mask and lets a potter or admin propose an alternative mask assisted by browser-side ML tools.

Without this component, none of those issues can accumulate human-provided ground-truth labels. It is the single prerequisite for any supervised labeling flow in the milestone.

This issue extracts the editor requirement from #420 into a standalone, reusable component so it can be built, tested, and iterated independently of the annotation-persistence logic.

Proposed Solution

New component: web/src/components/MaskEditor.tsx

Props

type MaskEditorProps = {
  // Source image to annotate
  imageUrl: string;
  imageWidth: number;
  imageHeight: number;

  // Optional pre-filled candidate mask: RGBA PNG with RGB zeroed, alpha = foreground confidence
  // (matches the CropRun.mask_asset format from #421)
  candidateMask?: string | null;  // Cloudinary URL or data URI

  // Called when the user commits a mask; receives the edited mask as a canvas ImageData or blob
  onCommit: (mask: Blob) => void;
  onCancel: () => void;

  // Optional: disable tools not yet implemented
  disabledTools?: ToolName[];
};

Required tools (all must be present; pixel-by-pixel brush is NOT the primary path)

Tool	Description	Implementation path
Pre-fill	Load candidateMask onto the canvas as the starting state	Canvas alpha composite
Brush + eraser	Adjustable-radius soft/hard edge	Canvas 2D
Polygon edit	Add/move/delete vertices on the mask contour	Canvas 2D + hit testing
Flood fill	Click inside a region to add/subtract from mask, with tolerance slider	Canvas 2D (no ML needed)
GrabCut	Drag rect + optional foreground/background scribble hints → refined mask candidate	opencv.js or server-side assist
Contour snap	Pull a polygon vertex toward the nearest strong image edge within a radius	opencv.js or server-side assist

Browser-ML topology decision

GrabCut and contour snap are OpenCV-shaped operations. Two viable paths — the implementer must prototype both for GrabCut on a representative pottery image (busy background, ~4MP source) and pick based on measured latency and bundle impact:

1. WASM in-browser (opencv.js or trimmed wasm build): instant feedback, no round-trip. Costs ~8 MB initial payload; memory ceilings on large images.
2. Server-side assist API (POST /api/annotations/assists/{flood-fill,grabcut,contour-snap}): leverages Python OpenCV already in the build graph; one implementation. Costs one round-trip per click.

Document the chosen topology in docs/agents/glaze-domain.md so downstream issues (#420, #418) don't relitigate it.

State shape

The editor has multiple coupled fields (mask draft, undo stack, active tool, tool params, dirty flag) advanced by multiple event sources. Per react-conventions/SKILL.md, use useReducer keyed on domain events (hydrate, tool_applied, prefill_received, commit_requested, cancelled) rather than a pile of useState calls.

Undo/redo: at minimum 20 steps; keyed on canvas snapshots (not deltas) for simplicity.

Layout

Full-screen MUI Dialog sized to viewport. Extract child components as needed to keep JSX nesting ≤ 4: MaskEditorToolbar, MaskEditorCanvas, MaskEditorFooter. MUI theme tokens only — no inline palette values.

Mask format in/out

• Input (candidateMask): RGBA PNG URL where alpha = foreground confidence. Alpha > 128 = foreground on load.
• Output (onCommit): same RGBA PNG format — RGB zeroed, alpha = foreground/background (binary or soft edge at implementer's discretion). This matches CropRun.mask_asset and CropAnnotation.mask formats throughout the milestone.

Acceptance Criteria

• MaskEditor component renders source image with candidate mask overlaid (alpha-composited).
• All six tools wired in the toolbar: pre-fill, brush + eraser, polygon edit, flood fill, GrabCut, contour snap.
• Browser-ML topology (wasm vs. server-side) prototyped for GrabCut on a representative image; chosen path implemented and documented in docs/agents/glaze-domain.md.
• useReducer state shape with named domain events; no bare useState for mask or tool state.
• Undo/redo working across all tools (minimum 20 steps).
• onCommit emits an RGBA PNG Blob (RGB zeroed, alpha = mask); onCancel closes without side effects.
• Component is self-contained: accepts image URL + optional candidate mask URL; has no direct knowledge of CropRun, CropAnnotation, or any other API model. Callers wire those.
• Frontend tests cover: pre-fill loads candidate mask, each tool mutates the mask state, undo reverts, commit emits correct blob, cancel fires without mutation.
• Storybook stories for: empty state (no candidate), pre-filled candidate, each tool active.

Out of Scope

• Persisting the mask to any API endpoint — callers own that (see #420, #421).
• The padding preview slider — that belongs in #420 where crop_padding_fraction is defined.
• The "(re-)run backend" dropdown that lets annotators swap the candidate from a different CropRun — that also belongs in #420.
• Any server-side model changes.

Dependencies

• No upstream code dependencies within this milestone — can be built in parallel with #421.
• If the server-side assist topology is chosen, new endpoints (POST /api/annotations/assists/) must be added — those can be stubbed initially and filled in before the acceptance criteria are met.

Blocks

• #420 — Supervised Crop Annotation UI + Per-Potter Padding — the annotation UI mounts <MaskEditor> as its core surface
• #418 — A/B Routing & Comparison Dashboard — disagreement re-labeling queue uses the same component
• By chain: #415 → #416 → #417 → #418 all depend on training labels that flow through this component

Milestone Cross-References

Part of milestone #2 — Custom Pottery Crop Model.

[shaoster]

Agent Implementation Plan

Approach

Topology decision (resolved): GrabCut and contour-snap run WASM in-browser via @opencvjs/web, which is already installed. No new server-side assist endpoints needed.

New dependency: react-canvas-masker — provides brush/eraser/undo-redo as a useMaskEditor hook. Handles the low-level pointer-event drawing layer so we don't reimplement canvas painting from scratch.

---

Files

File	Purpose
web/src/components/MaskEditor.tsx	Main component (Dialog shell + child wiring)
web/src/components/maskEditorState.ts	useReducer state, action types, reducer
web/src/components/MaskEditorToolbar.tsx	Tool palette + radius/tolerance sliders
web/src/components/MaskEditorCanvas.tsx	Canvas rendering + pointer events
web/src/components/MaskEditorFooter.tsx	Commit/Cancel buttons + status
web/src/components/__tests__/MaskEditor.test.tsx	Unit tests

---

Canvas Architecture

Two stacked canvases (position: absolute) over a CSS background-image source:

1. Mask canvas — the editable layer. react-canvas-masker's useMaskEditor drives brush/eraser pointer events. Alpha channel is the live mask (255=foreground, 0=background).
2. Overlay canvas — ephemeral drawing only (polygon vertices/edges, GrabCut rect drag). Never included in output blob.

---

Tool Implementations

Tool	Implementation
Pre-fill	Load candidateMask URL → draw to offscreen canvas → threshold alpha > 128 → paint onto mask canvas
Brush + eraser	useMaskEditor from react-canvas-masker; brush=alpha 255, eraser=alpha 0
Flood fill	Scanline flood fill on mask canvas ImageData (pure Canvas 2D). Tolerance = color distance from seed pixel on source image.
Polygon edit	Vertices in reducer state. Overlay canvas draws edges + handles. Click=add, drag=move, right-click=delete. On close: fill polygon onto mask canvas.
GrabCut	User drags rect on overlay → "Run GrabCut" → resize source to ≤512px → cv.grabCut() via @opencvjs/web → upscale result → apply to mask canvas
Contour snap	On polygon vertex drag: cv.Canny() on source image region near cursor, snap to nearest strong edge pixel within radius

---

State Shape (maskEditorState.ts)

Domain-event useReducer pattern per codebase convention:

type MaskEditorAction =
  | { type: "hydrate"; snapshot: ImageData | null }
  | { type: "set_tool"; tool: ToolName }
  | { type: "set_brush_radius"; radius: number }
  | { type: "set_flood_fill_tolerance"; tolerance: number }
  | { type: "tool_applied"; snapshot: ImageData }
  | { type: "polygon_vertex_added"; x: number; y: number }
  | { type: "polygon_vertex_moved"; index: number; x: number; y: number }
  | { type: "polygon_vertex_deleted"; index: number }
  | { type: "polygon_committed"; snapshot: ImageData }
  | { type: "grabcut_rect_set"; rect: Rect | null }
  | { type: "undo" }
  | { type: "redo" }
  | { type: "assist_started" }
  | { type: "assist_succeeded"; snapshot: ImageData }
  | { type: "assist_failed"; error: string }
  | { type: "commit_requested" }
  | { type: "cancelled" };

Undo/redo: push canvas snapshot to undoStack before each tool_applied / polygon_committed / assist_succeeded. Cap at 20 steps.

---

Mask Format

• Input (candidateMask): RGBA PNG where alpha > 128 = foreground
• Internal: mask canvas alpha channel (255=foreground, 0=background)
• Output (onCommit): read mask canvas ImageData, zero RGB channels, canvas.toBlob("image/png") → blob

---

Layout

<Dialog fullScreen>
  <MaskEditorToolbar />        ← tool buttons + sliders
  <Box sx={{ position: "relative", flex: 1 }}>
    {/* source image as CSS background */}
    <canvas ref={maskCanvasRef}    style={{ position: "absolute" }} />
    <canvas ref={overlayCanvasRef} style={{ position: "absolute" }} />
  </Box>
  <MaskEditorFooter />         ← Commit / Cancel + status
</Dialog>

JSX nesting ≤ 4 per child component.

---

Security Surface

None. Pure frontend component — no auth, no API calls, no permission changes. Callers (#420, #418) own persistence and access control.

---

Out of Scope

• Server-side assist endpoints (POST /api/annotations/assists/)
• Padding slider → belongs in feat: supervised crop annotation UI with per-potter padding #420
• Backend-swap dropdown → belongs in feat: supervised crop annotation UI with per-potter padding #420
• Any backend model changes