Spinners for setup steps with --progress --batch#1943
Merged
Conversation
scottnemes
reviewed
Jun 19, 2026
| raise CheckpointReplayError(f'Error reading --batch file: {batch_path}: {e}') from None | ||
| if progress: | ||
| spinner = yaspin(text='replaying checkpoint', side='right', stream=sys.stderr) | ||
| spinner.start() |
Contributor
There was a problem hiding this comment.
This instance of spinner doesn't use with like the other, so if the outer try/except errors out after the spinner starts it won't be stopped
Contributor
Author
There was a problem hiding this comment.
Thanks, good catch!
rolandwalker
force-pushed
the
RW/more-batch-steps-before-progress-bar
branch
from
42efaf9 to
5f25b7a
Compare
For large --batch scripts (the very kind of SQL scripts for which the
progress bar is useful) the initial steps of pre-parsing the batch and
optionally replaying the checkpoint log can take substantial time.
Therefore, add text and spinners showing the stage and progress of the
setup steps.
A lightweight new dependency is added due to limitations in the
spinners from prompt-toolkit: the prompt-toolkit spinners must have a
known upper bound to which to iterate, and they leave behind random
"mid-spin" characters on completion like
pre-parsing batch \
which creates visual confusion about whether the step completed
successfully.
yaspin spinners can integrate with prompt-toolkit more deeply than is
done here, and could also for instance be used during long interactive
queries (though that particular usecase may have little value).
Another option would be spinners from the "rich" library, which we may
try in the future, if adopting "rich" for tabular output.
rolandwalker
force-pushed
the
RW/more-batch-steps-before-progress-bar
branch
from
5f25b7a to
7336214
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
For large
--batchscripts (the very kind of SQL scripts for which the progress bar is useful) the initial steps of pre-parsing the batch and optionally replaying the checkpoint log can take substantial time. Therefore, add text and spinners showing the stage and progress of the setup steps.A lightweight new dependency is added due to limitations in the spinners from
prompt-toolkit: theprompt-toolkitspinners must have a known upper bound to which to iterate, and they leave behind random "mid-spin" characters on completion likewhich creates visual confusion about whether the step completed successfully.
yaspinspinners can integrate withprompt-toolkitmore deeply than is done here, and could also for instance be used during long interactive queries (though that particular usecase may have little value).Another option would be spinners from the
richlibrary, which we may try in the future, if adoptingrichfor tabular output.Motivation: the pause before showing the progress bar was recently opaque to me when running a large
--batchscript of ~70M statements. It took several seconds to parse the script.Examples:
Checklist
changelog.mdfile.AUTHORSfile (or it's already there).