Skip to content

Conversation

@bodom0015
Copy link
Member

Problem

ChemScraper is our only AlphaSynthesis job that does not run in Kubernetes

Pairing our new chemscraper Python client with our existing current Python post-processing, we should be able to create a job container to submit ChemScraper requests

Approach

  • feat: created a proof-of-concept for running ChemScraper jobs in Kubernetes

How to Test

  1. Checkout and run this branch locally
  2. Start the test job: docker compose up --build
    • You should see the Docker image build before starting up
    • You should see the job start up and automatically submit a test file to the ChemScraper API (currently uses staging)
  3. Wait for the job to complete
    • You should see the container stop after the job output CSV is written
  4. Check the outputs/ folder
    • You should see the contents of the ChemScraper ZIP response has been unzipped into the output files
    • You should see a JOBID-results.csv has been created in this folder as well

@bodom0015 bodom0015 merged commit da83a34 into main Apr 3, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants