Skip to content

jarzul/use dce progress api for progress bar#88

Merged
JulienArzul merged 5 commits intomainfrom
jarzul/use-dce-progress-api-for-progress-bar
Apr 2, 2026
Merged

jarzul/use dce progress api for progress bar#88
JulienArzul merged 5 commits intomainfrom
jarzul/use-dce-progress-api-for-progress-bar

Conversation

@JulienArzul
Copy link
Copy Markdown
Contributor

What?

Following up on #36, this is refactoring the implementation reading the DCE logs to actually use the ProgressCallback API from DCE (as originally intended in the previous PR)

How?

I've reused the initial implementation from Mateus in the other PR but I've made a couple of changes.

1. Rich console used for logging

instead of relying on a complex logic trying to go through all loggers to make them compatible with the rich Progress (and then restoring the original logger's stdio), I'm now relying fully on Rich within the project.

A new rich_console singleton has been added and both the logging configuration and the cli_progress are using it. That means that we don't need to do anything for the Progress to behave correctly in cli_progress: the loggers are already printing on the rich console.

2. Separate event handling and rendering

To make the code a bit more understandable (IMO), I've separated it into two:

  • creating a progress state based on the events received: the state is updated by each received event
  • rendering that state with the rich Progress

…ogging configuration and when creating a rich Progress

This allows for all logs to go through rich and making regular logs behave correctly out-of-the box with the rich Progress bar
…e from the events received and the actual rendering using rich's Progress bar
@JulienArzul JulienArzul requested a review from a team April 1, 2026 12:33
@JulienArzul
Copy link
Copy Markdown
Contributor Author

Example with 30 sqlite datasources to build:

Screen.Recording.2026-04-01.at.14.40.28.mov

In this example, the build for each datasource is quite fast and the only thing that takes some time is the embedding step for each datasource => so we almost only see the progress waiting at 33% of each datasource (the first step is the actual introspection which is super fast on sqlite because there is no profiling done and the last step is storing everything in the DB which is also very fast)

@JulienArzul JulienArzul requested a review from gasparian April 1, 2026 12:43
@JulienArzul JulienArzul merged commit e1e1f00 into main Apr 2, 2026
6 checks passed
@JulienArzul JulienArzul deleted the jarzul/use-dce-progress-api-for-progress-bar branch April 2, 2026 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants