Feature to ingest template images from surveys for O4 galaxies

[crpellegrino]

This PR allows for automatic ingestion of template images from SDSS (and soon, PS1 and DECam) for galaxies that are observed by LCO during LVK O4. Three new files--ingesttemplates.py, runo4templateingest.py, and rungwgalaxydiff.py--handle the template image ingestion, PSF generation, cosmic ray removal, and background subtraction. Each can be run from the command line by passing a target ID and filter using the --targetid and -f flags. To get the template ingestion to work without having LCO images obtained beforehand, I had to rewrite those existing stages to accept either a list of files OR both a target ID and filter. Currently I've tested this for SDSS ingestion and reduction, and once the PS1 and DECam ingestion features are merged those can be tested too.

During the process I also discovered what I think is a bug with the pipeline installation instructions. Step 9 says to export LCOSNDIR=/your/data/directory but when ingesting reference images this path didn't work and threw an error. I had to set LCOSNDIR=/your/data/ instead to get the ingestion to work. I'm not sure if this is a bug that's specific to reference image ingestion or if it's a bigger issue, but I've left it alone in the code for now.

[abostroem]

Thanks for the PR @crpellegrino. Could you please provide a set of instructions for how to test this PR and what a successful test would look like? And before merging, could you update the manual.md file with documentation on how to use these features?

[crpellegrino]

I tested this by running the new ingesttemplates.py, runo4templateingest.py, and rungwgalaxydiff.py commands from the command line. ingesttemplates.py doesn't have any special requirements, but the other two commands need some entries in the new o4_galaxies table we're adding to the database. You can populate that manually using the ra, dec, and targetid of a pre-existing target. A successful test is if templates from SDSS are ingested in g,r, and i and if the pipeline stages up to and including the -s diff stage are executed (the outputs from these commands should be written to log files rather than to the terminal).

I don't think anything has to be added to the manual because these are all changes that will only be run via cron in production at LCO. Users won't have to interface with these new commands unless they want to ingest survey templates programmatically for whatever reason. Otherwise these changes shouldn't have any effect on the normal pipeline workflow.