feat(media): add detailed scrape progress status

[wizzomafizzo]

Summary

• add explicit scrape lifecycle state and error fields to media.scrape.status/media.scraping
• add per-system progress plus whole-run step progress fields
• update scraper progress publishing, tests, and API docs

Validation

• go test ./pkg/api/methods ./pkg/api/notifications ./pkg/database/scraper/gamelistxml

Summary by CodeRabbit

• New Features

  • Scraping status now reports explicit lifecycle states (idle, running, paused, completed, cancelled, failed), per-system progress, and step-level progress (total/current step and display).

• Documentation

  • API docs and examples updated to show the richer scraping status and notification payloads and guidance for UI progress fields.

• Tests

  • Expanded test coverage to validate new state, step, and per-system progress fields in status and notification flows.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d53d5926-9f52-42e2-a9e2-7cb8a4d2902d

📥 Commits

Reviewing files that changed from the base of the PR and between 0e1d0b6 and 99700a6.

📒 Files selected for processing (2)

• pkg/database/scraper/gamelistxml/scraper.go
• pkg/database/scraper/gamelistxml/scraper_test.go

🚧 Files skipped from review as they are similar to previous changes (1)

• pkg/database/scraper/gamelistxml/scraper.go

---

📝 Walkthrough

Walkthrough

Enriches scrape lifecycle with explicit State values and step-level progress: extends progress events/types, refactors scraper loop to emit TotalSteps/CurrentStep everywhere, updates API handlers to populate State/currentSystem, expands tests to assert new fields, and updates documentation and notification examples.

Changes

Scrape Lifecycle State and Step Progress

Layer / File(s)	Summary
Scrape state constants and response types pkg/database/scraper/scraper.go, pkg/api/methods/media_scrape.go, pkg/api/models/responses.go	Adds TotalSteps/CurrentStep to ScrapeUpdate. Introduces named state constants and the ScrapeSystemProgressResponse type. Expands ScrapingStatusResponse with optional state, error, totalSteps, currentStep, currentStepDisplay, and currentSystem.
Scraper loop progress propagation pkg/database/scraper/gamelistxml/scraper.go	scrapeLoop computes totalSteps, adds sendUpdate to consistently set TotalSteps/CurrentStep, threads currentStep into resume checks and companion processing, and ensures all cancellation/fatal/termination branches emit step-aware updates.
API state lifecycle handlers and helpers pkg/api/methods/media_scrape.go	Adds helper functions to derive human-readable current-system display and nullable step pointers, refactors status construction into scrapingStatusFromUpdate, sets initial State on start, normalizes State on status reads, and computes terminal State for completion/cancellation/failure.
Test assertions for state and step fields pkg/api/methods/media_scrape_test.go, pkg/api/notifications/notifications_test.go, pkg/database/scraper/gamelistxml/scraper_test.go	Updates tests to assert State values (running/cancelled/paused/failed), adds TestHandleMediaScrape_EmitsFatalStatus, enhances progress tests to include TotalSteps/CurrentStep and to verify CurrentSystem payloads, and adds a scrape-loop test for pause+cancel preserving per-system progress.
API and scraper documentation for new state and progress fields docs/api/methods.md, docs/api/notifications.md, docs/scraper.md	Document explicit state semantics (idle/running/paused/completed/cancelled/failed), new step progress fields and currentSystem object, and update example payloads; clarify totalScraped derivation in scraper docs.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• ZaparooProject/zaparoo-core#858: Modifies the gamelistxml scraper’s scrapeLoop and companion processing used by these step/state changes.

Poem

🐰 Hop, hop—each scrape now keeps its state,
From idle hush to running's steady gait.
Steps counted, systems shown with care,
Progress blooms like carrots everywhere! 🥕✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat(media): add detailed scrape progress status' accurately and specifically summarizes the main change: adding enhanced progress tracking fields and lifecycle state to media scraping operations.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/media-scrape-status-progress

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/database/scraper/gamelistxml/scraper.go`:
- Line 560: The call waitForResume(system.ID, currentStep, 0, 0, 0) passes
zeroed counters causing the pause/cancel terminal update to drop previously
accumulated progress; replace the three zero literals with the actual
accumulated progress variables used in this scope (e.g., accumulatedProcessed,
accumulatedTotal, accumulatedErrors or whichever names track total progress),
ensure those variables are kept up-to-date before the call, and propagate them
into waitForResume so the emitted "Done" update preserves prior progress for
system.ID and currentStep.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 18c26a32-5e00-4ca7-a258-c44636dc37b3

📥 Commits

Reviewing files that changed from the base of the PR and between 0316142 and 0e1d0b6.

📒 Files selected for processing (9)

• docs/api/methods.md
• docs/api/notifications.md
• docs/scraper.md
• pkg/api/methods/media_scrape.go
• pkg/api/methods/media_scrape_test.go
• pkg/api/models/responses.go
• pkg/api/notifications/notifications_test.go
• pkg/database/scraper/gamelistxml/scraper.go
• pkg/database/scraper/scraper.go

[sentry]

Codecov Report

❌ Patch coverage is 63.57616% with 55 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/database/scraper/gamelistxml/scraper.go	45.07%	37 Missing and 2 partials ⚠️
pkg/api/methods/media_scrape.go	80.00%	10 Missing and 6 partials ⚠️

📢 Thoughts on this report? Let us know!