feat: Webhook / Event Stream — real-time notifications for integrations

[SimplicityGuy]

Summary

Expose webhook registration and Server-Sent Events (SSE) endpoints so users and developers can subscribe to platform events: extraction completions, sync results, gap analysis changes, and more.

Motivation

The platform generates valuable events (extraction complete, sync finished, new data available) that external systems could react to. A developer might want to trigger a script when their collection sync finishes, or a home automation system could display "new music data available" on a dashboard.

Proposed Endpoints

POST   /api/webhooks              — register a webhook URL
GET    /api/webhooks              — list registered webhooks
DELETE /api/webhooks/{id}         — remove a webhook
GET    /api/events/stream         — SSE endpoint for real-time events

Event Types

Event	Payload	Trigger
extraction.started	version, entity types	Extractor begins processing
extraction.completed	version, record counts, duration	Extractor finishes
sync.completed	user_id, items_synced, errors	Collection sync finishes
sync.failed	user_id, error_message	Collection sync fails
gaps.updated	user_id, entity_type, entity_id	Gap analysis results change after sync

Webhook Delivery

• POST to registered URL with JSON payload + HMAC-SHA256 signature header
• Retry with exponential backoff (3 attempts)
• Per-user webhook limit (e.g., 5 webhooks)
• Webhook secret generated on registration for signature verification

SSE Stream

GET /api/events/stream
Authorization: Bearer <jwt>

event: sync.completed
data: {"user_id": "abc", "items_synced": 150, "duration_s": 12.3}

event: extraction.completed
data: {"version": "20260301", "artists": 8500000, "releases": 16000000}

Implementation Notes

• Store webhook registrations in PostgreSQL
• Use Redis pub/sub to bridge internal events to SSE connections
• Dashboard already uses WebSocket for real-time updates — SSE is a simpler alternative for integrations
• HMAC signing prevents spoofed webhook deliveries

Complements

• feat: Notifier Service — watchlists and alerts for new music data #86 — Notifier service handles user-facing alerts (email, push); webhooks serve developer/integration use cases
• feat: Insights Service — precomputed analytics and music trends #85 — Insights service could emit events when new analytics are computed

Acceptance Criteria

• Webhook CRUD endpoints with per-user limits
• HMAC-SHA256 signature on webhook deliveries
• Retry with exponential backoff (3 attempts)
• SSE endpoint streams events in real time
• At minimum: sync.completed, extraction.completed events
• Webhook secret generated and returned on registration

[SimplicityGuy]

Update: The Insights service (#85) is now live and could emit events for the event stream (e.g., computation complete, new insights available).

[SimplicityGuy]

Status Update — 2026-03-25

Not started. No implementation or design work has been merged to main for this feature.