Skip to content

[READY FOR REVIEW] Implement Database-Backed Ingestion Progress Bar#68

Merged
nitesh32 merged 1 commit into
pre-releasefrom
feat/progressCli
May 25, 2026
Merged

[READY FOR REVIEW] Implement Database-Backed Ingestion Progress Bar#68
nitesh32 merged 1 commit into
pre-releasefrom
feat/progressCli

Conversation

@bharatsachya
Copy link
Copy Markdown
Collaborator

What changed

  • Created DbProgressReporter: Added DbProgressReporter.ts to track, throttle (250ms), and write real-time ingestion progress to MongoDB or SQLite via the knowledgeDb abstraction.
  • Defaulted Worker Configuration: Configured the ingestion workers in packages/ingest-github/src/index.ts to use dbProgressContextFactory by default, keeping public registration signatures clean and localized.
  • Improved UI Liveness: Prevented the progress bar from resetting to 0% after the file analysis phase completes (during folder analysis and database indexing phases) by maintaining the progress at 100% of analyzed files.
  • Fixed File Count Display: Updated reposRoute.ts in the server to fall back to status.totalFiles for reporting the fileCount value, ensuring the CLI prints the correct number of files indexed (e.g. 17 files) rather than 0 files upon completion.

Why

The CLI depends on polling the server's repos REST endpoint to show progress and file counts. However, since the open-source client clones to disk directly and does not write raw file archives to the database, fileCount returned 0 and progress fields stayed undefined (because workers defaulted to nullProgressContextFactory).

This PR resolves both issues without requiring any changes to public registration APIs or server-boot configurations.


How to test

  1. Boot up the backend infrastructure and the server:
    bun run bytebell boot

bytebell index repo

@bharatsachya bharatsachya changed the base branch from main to pre-release May 25, 2026 08:56
@bharatsachya bharatsachya requested a review from Dead-Bytes May 25, 2026 08:57
@bharatsachya
Copy link
Copy Markdown
Collaborator Author

Against #66

@bharatsachya bharatsachya added enhancement New feature or request Ready for Review Ready for review from reviewers labels May 25, 2026
@nitesh32 nitesh32 merged commit 239a7e6 into pre-release May 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Ready for Review Ready for review from reviewers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants