Skip to content

Add a storage saving mode to XMM processing in DAXA #429

@DavidT3

Description

@DavidT3

Though DAXA will at some point be able to stream data from the S3 bucket, there will still be a necessity to store some data locally (i.e. on a Fornax home directory), if raw XMM data are processed into cleaned event lists etc.

I've always erred on the side of keeping everything, but an effort should be made towards retaining only essential outputs to save storage (expensive on AWS).

First target is XMM processing - don't need to keep the non-final event lists, that could save a decent fraction of the storage consumed per ObsID (though observation length will determine the absolute size of that saving).

Metadata

Metadata

Assignees

Labels

refinementIf a feature has already been implemented, and works, but could do with another pass to improve it.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions