feat: Engagement Insights workflow

[walkerfrankenberg]

Overview

New generateEngagementInsights() workflow that explains why specific moments in a video have high or low viewer engagement. Combines Mux engagement primitives (hotspots, heatmaps), visual frames (via shots API with thumbnail fallback), transcripts, and AI to produce structured per-moment explanations and overall engagement analysis.

Also extracts computeHeatmapStatistics() and computeHeatmapPercentile() as public primitives so other workflows can compose with engagement data.

What was changed

• src/workflows/engagement-insights.ts (new) — Full workflow: parallel data fetching via Promise.allSettled, shots integration with 30s timeout and thumbnail fallback, hotspot deduplication, deterministic high/low type computation from heatmap average, percentile normalization, hotspotIndex-based re-association with AI output, prompt overrides, skipShots option, withRetry on AI generation.

• src/primitives/heatmap.ts — Extracted computeHeatmapStatistics() (average, peak, lowest, significant drops with consecutive-drop merging) and computeHeatmapPercentile() as public exports. Fixed heatmap API response shape (data wrapper) and default timeframe.

• src/primitives/hotspots.ts — Matching timeframe default fix.

• tests/unit/engagement-insights.test.ts (new) — Unit tests for helper functions: heatmap statistics, percentile computation, shot-to-hotspot mapping, transcript segment extraction, hotspot deduplication.

• tests/integration/engagement-insights.test.ts (new) — Integration tests: all 3 providers, all 3 insight types, shots fallback paths, skipShots, audio-only, empty data errors, upstream error propagation.

• tests/helpers/mock-engagement-data.ts — Added shot mocks.

• docs/API.md, docs/WORKFLOWS.md, README.md — Documentation for the new workflow.

Suggested review order

1. docs/WORKFLOWS.md — Feature overview and usage examples
2. src/workflows/engagement-insights.ts — Core implementation (types → schemas → prompts → helpers → main workflow)
3. src/primitives/heatmap.ts — Extracted statistics and percentile primitives
4. tests/unit/engagement-insights.test.ts — Unit tests for helpers
5. tests/integration/engagement-insights.test.ts — Integration tests with mocked primitives

---

Note

Medium Risk
Adds a new AI-driven workflow that orchestrates multiple Mux Data/Video fetches plus LLM structured output, and changes engagement primitive defaults/response parsing; failures here can affect runtime behavior and downstream consumers of engagement endpoints.

Overview
Adds a new generateEngagementInsights() workflow that explains why specific moments have high/low viewer engagement by combining Mux engagement hotspots + heatmap with transcript text and visual context (shots when available, otherwise thumbnails, with optional skipShots).

Updates engagement primitives to match the current Mux Data API response shape (data wrapper) and changes default timeframe behavior to "7:days" for both heatmap/hotspots; refines askQuestions prompting to more strictly skip irrelevant questions.

Includes new unit/integration tests and documentation updates (README, docs/API.md, docs/WORKFLOWS.md) covering the new workflow and its options/return type.

^{Written by Cursor Bugbot for commit 791146c. This will update automatically on new commits. Configure here.}

[snyk-io]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[walkerfrankenberg]

So the WorkflowDevKit tests were failing because the workflow hits real Mux engagement APIs, and the test asset has no engagement data.

The regular integration tests pass because vi.mock() works in the normal vitest process. But the WorkflowDevKit spins up a separate Nitro server that doesn't see the mocks.

The test asset just doesn't have engagement data. For the WorkflowDevKit tests, we either need an asset that has real engagement data, or we need to accept that this test can't run without one. For now, we're skipping this and relying on the integration tests, but we probably want a better story for this going forwards

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review.

_{Tip: disable this comment in your organization's Code Review settings.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[daniel-hayes]

very excited about this one!