Conversation
|
Someone is attempting to deploy a commit to a Personal Account owned by @neokd on Vercel. @neokd first needs to authorize it. |
|
Can the proxy be made like an parameter? Like if user set as True then proxy works or else it tries to scrape without proxy.? |
Are you referring to a user-agent a browser would use? |
|
No, for now we were thinking that proxy is getting complicated so if we allow user to pass an argument that they want to use scrapper or not. |
I see. I'll look into adding that option for the scraper. |
|
@Bchass looks good I'll review it fully and merge it. Thinking to publish the scrapper as package on Pypi? |
|
Is this project being maintained anymore? |
|
@Bchass yeah the project is maintained. There was merge conflict so i didn't merge this PR. |
Description
Fixed a bug and switched over to a
dequeinstead of a listRelated Issues
#112
Changes Made
load_proxy_list()doesn't rely on an absolute path anymore, this wasn't working as intended.rotate_proxy()Switched over to a deque for better efficiency when rotating proxiesScreenshots (if applicable)
N/A
Checklist
Please review and check the following before submitting your pull request:
Additional Notes
I would like to add test cases for this