Search engine for scraping data about COVID tests ordered by companies for employees
- Download client secrets json file from the google cloud console (https://console.cloud.google.com/apis/credentials/oauthclient/702218053502-fcrju4976lt0p1dntbln2qdolo72qjki.apps.googleusercontent.com?authuser=1&project=covid-corporate--1589232879130)
- Rename it as
client_secrets.jsonand move it to the root directory of this project - Install python 3.8+
- Optional: Create a virtual environment and activate it
- Install dependencies via
pip install -r requirements.txt - Set the following environment variables:
- CSE_ID = the ID of the customized search engine
- CUSTOM_SEARCH_API_KEY = the custom search API key
- Run the script
python main.py. The program will prompt you to sign in with a google account, make sure you use an account that has access to the COVID drive (specifically the data team folder)
Refer to the corporate_search.sh documentation in the scripts repo.