Skip to content

Add audio & embed resources (MP3 upload, YouTube, Google Drive)#15

Merged
pedrobritx merged 1 commit into
mainfrom
claude/audio-youtube-imports-f2r9h
May 31, 2026
Merged

Add audio & embed resources (MP3 upload, YouTube, Google Drive)#15
pedrobritx merged 1 commit into
mainfrom
claude/audio-youtube-imports-f2r9h

Conversation

@pedrobritx
Copy link
Copy Markdown
Owner

What

Lets teachers attach audio/video resources to a board as canvas objects, alongside the existing image/PDF imports:

  • MP3 (audio) upload → stored in Supabase Storage, played with a native HTML5 player.
  • YouTube embed → paste a URL, full resizable iframe player.
  • Google Drive audio → paste a shared link, embedded via Drive's preview iframe.

How

Konva renders to <canvas>, which can't host <audio>/<iframe>. So each resource is a new YEmbed shape drawn as a selectable Konva card (EmbedRenderer) with a live HTML player floated over the stage (MediaOverlayLayer), tracking pan/zoom via the same world→screen transform as the existing text-edit overlay.

  • The overlay is click-through until the card is selected (pointer-events toggle), so Konva still drives select/drag/resize/z-order/lock with no changes to SelectTool or the Transformer.
  • MP3 reuses the existing asset pipeline: AssetKind gains "audio"; importFiles classifies audio/* and places a card (no bitmap/PDF decoding). The player resolves the public Storage URL at render time.
  • YouTube/Drive: parseEmbedUrl() normalises watch?v= / youtu.be / shorts / embed and Drive file/d/<id> / open?id= links to canonical iframe URLs (unit-verified). assetStore.insertEmbed() drops a card at the viewport centre.
  • UI (AppMenu): import accepts audio/*; a new "Embed YouTube / Google Drive…" popover takes a URL with inline validation.
  • "embed" is wired through shapeBounds/translateShape, ShapesLayer, TransformLayer, and SelectionInspector (the YShape union is exhaustive, so tsc confirms every site is handled).
  • Migration 0003 widens the assets.kind CHECK to allow 'audio'.

Sync / persistence

YEmbed is an ordinary Yjs shape, so it syncs over Supabase Realtime and persists in snapshots/IndexedDB automatically — no new sync plumbing. Each viewer controls their own playback (no shared transport position in this pass).

Verification

  • pnpm -r typecheck passes (all 5 packages).
  • pnpm build passes.
  • parseEmbedUrl unit-checked against watch/youtu.be/shorts/embed/Drive URLs and negative cases (non-YouTube/Drive → inline error).

Manual (recommended before merge, needs Supabase configured)

  1. MP3: Import → pick an .mp3 → card appears at centre; select it → HTML5 player plays; drag/resize/rotate/lock behave like an image; pan/zoom keeps the player pinned.
  2. YouTube: Embed → paste a watch?v= / youtu.be URL → video iframe embeds.
  3. Google Drive: paste a link-shared audio file → Drive preview iframe plays.
  4. Bad URL → inline error, nothing inserted.
  5. Sync: open in a second window → card appears for the peer; reload → persists.

Notes / out of scope

  • No synchronised playback position across viewers.
  • YouTube uses the standard full-video iframe (per request); a compact/audio-only mode can layer onto the same YEmbed later.
  • Google Drive files must be shared "anyone with the link" (surfaced in the UI).

https://claude.ai/code/session_012m6DUY6q4Vk2WuUh9GtkpA


Generated by Claude Code

Teachers can attach audio/video resources to a board as canvas objects,
alongside images and PDFs.

- New `YEmbed` shape kind (audio | youtube | gdrive) with a Konva card
  (EmbedRenderer) plus a DOM player floated over the stage
  (MediaOverlayLayer), since Konva's canvas can't host <audio>/<iframe>.
  The overlay is click-through until the card is selected, so Konva still
  drives select/drag/resize unchanged.
- MP3 upload reuses the existing asset pipeline: AssetKind gains "audio";
  importFiles classifies audio/* and places a player card (no decoding).
- YouTube/Drive embeds via pasted URL: parseEmbedUrl() normalises watch/
  youtu.be/shorts/embed and Drive file/open links to canonical iframe URLs;
  assetStore.insertEmbed() drops a card at the viewport centre.
- AppMenu: accept audio/* in import; new "Embed YouTube / Google Drive" URL
  popover with inline validation.
- Wire "embed" through shapeBounds/translate, ShapesLayer, TransformLayer,
  and SelectionInspector so transform/move/z-order/lock all apply.
- Migration 0003 widens the assets.kind check to allow 'audio'.

https://claude.ai/code/session_012m6DUY6q4Vk2WuUh9GtkpA
@pedrobritx pedrobritx marked this pull request as ready for review May 31, 2026 23:30
@pedrobritx pedrobritx merged commit 2df9e11 into main May 31, 2026
4 checks passed
@pedrobritx pedrobritx deleted the claude/audio-youtube-imports-f2r9h branch May 31, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants