Open
Conversation
…een requests to prevent being blocked 1. adds the -R flag 2. Should fix https://github.com/scoliono/archiveripper/issues/1 and adds the -nt flag
Because we no longer overwrite files without the user asking explicitly for it (-R flag) The prompt also got in the way of running the script non-interactively
It's more informative to log the actual filename, which includes the page number. I also feel that gauging the progress is easy enough with (N/N) to make a percentage indicator unnecessary. Changed from: 12% (1/8) done 25% (2/8) done 37% (3/8) done 50% (4/8) done 62% (5/8) done 75% (6/8) done 87% (7/8) done 100% (8/8) done To: Got ./OL370939M/100.jpg (1/8) Got ./OL370939M/101.jpg (2/8) Got ./OL370939M/102.jpg (3/8) Got ./OL370939M/103.jpg (4/8) Got ./OL370939M/104.jpg (5/8) Got ./OL370939M/105.jpg (6/8) Got ./OL370939M/106.jpg (7/8) Got ./OL370939M/107.jpg (8/8) (The command used to generate these logs was: `python3 ripper.py OL370939M -s 100 -e 107 -S 10`)
Contributor
Author
|
@scoliono merge? |
fixes scoliono#9
Owner
|
This does not appear to totally circumvent Archive.org's rate limiting, from my testing. Around 100 pages or so, you start downloading 5 KB HTML documents instead of images. |
Owner
|
If you're too persistent with the requests, it looks like you can also get this traceback: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@scoliono here are some changes for review. 8569763 should fix #1.
Also your email visible in
git logis very nice.