Create persistent S3 storage

[nilsmechtel]

No description provided.

[nilsmechtel]

@oeway When the server is stopped and restarted it does not detect and list artifacts from the previous session. Do you have an idea why?

minio_config_dir and minio_workdir do not get deleted anymore when a server is stopped. But since previous artifacts are not detected, this currently leads to an accumulation of copies of the exact same data for each restart.

The data server can be started like this: python -m bioengine.datasets --data-import-dir /path/to/data

--workspace-dir is by default set to ~/.bioengine.

This is the file structure created in proxy_server.py:

~/.bioengine/datasets
├── bin
├── bioengine_current_server
├── config                                          # minio_config_dir
└── s3                                              # minio_workdir

[nilsmechtel]

Could it have something to do with this?

WARNING:redis-store:Using in-memory SQLite database for event logging, all data will be lost on restart!