Skip to content

feat(onboarding): Attach images in onboarding chat for vision-based understanding #65

Description

@TemaDeveloper

Size: L — headline onboarding improvement.

Why

Onboarding is a pure text interview (ONBOARDING_SYSTEM_PROMPT, conversational-onboarding.tsx — the composer is a single text input, no attach affordance). People can describe their life far faster by showing it: a photo of their paper planner, a screenshot of a budgeting spreadsheet, a gym program, a class timetable, a meds list, a whiteboard of goals. Letting them attach images makes the derived life-profile richer and the generated sections fit better on the first try.

Scope

  1. Client (src/components/onboarding/conversational-onboarding.tsx): add an attach button + thumbnail previews + remove; allow paste/drag-drop; send image refs alongside the text turn.
  2. Upload: reuse the existing private-Blob path (/api/notes/upload: image-only, 10MB, ownership-scoped serving via /api/notes/blob) — or a parallel onboarding/<userId>/... namespace. Don't send raw base64 through the chat body.
  3. API (src/app/api/profile/chat/route.ts): accept images on the turn; pass them to the vision-capable extraction + reply calls (depends on multimodal callAI — see the multimodal-callAI issue).
  4. Extraction (src/lib/profile/facet-extract.ts): feed images so facets are derived from what's shown, not just typed.
  5. Update ONBOARDING_SYSTEM_PROMPT to invite "type or show me" and to reason over attached images.

Notes

  • Vision inference on multiple images is slow — respect/raise maxDuration (currently 60s) and stream or show progress.
  • Privacy: onboarding images can be sensitive (finances, health) — keep them private-access, scoped to the user, and consider deleting them after profile extraction.

Depends on: multimodal callAI. Pairs with: seed-from-artifact, confirm-understanding.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions