This repository was archived by the owner on Mar 6, 2026. It is now read-only.
This repository was archived by the owner on Mar 6, 2026. It is now read-only.
Feature: Concurrent scraping to speed up source gathering #49
Description
Currently finding sources is pretty slow and even more when some hang for longer due to network issues. There is no function to scrape multiple sources at a time.
The solution would be to add a runAllProvidersConcurrent function which runs just like runAllProviders except instead of sequentially scraping sources it is able to scrape X amount at a given time (specified in RunnerOptions defaults to 3)