QCDIS/Hackathon25
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
{
"cells": [
{
"cell_type": "markdown",
"id": "45aa531f-ad0a-4b74-b4d1-44919bb175f4",
"metadata": {},
"source": [
"# BioDT School & Hackathon 2025\n",
"\n",
"This is the central repository of training material that will be used during the BioDT School and Hackathon 2025.\n",
"\n",
"\n",
"## NaaVRE Documentation\n",
"\n",
"The documentation of the NaaVRE project can be found [here](https://naavre.net/).\n",
"\n",
"\n",
"## Saving files in NaaVRE workflows with MinIO\n",
"\n",
"## MinIO Setup\n",
"\n",
"1. Open [MinIO login](https://scruffy.lab.uvalight.net:9001/login)\n",
"2. Click on **\"Login with SSO (vre)\"**\n",
"\n",
"\n",
"\n",
"**⚠ Warning:** \n",
"This service is intended for testing and development purposes. \n",
"It is maintained by the LifeWatch VLIC team as “best effort.” \n",
"- There is **no guarantee** that your data is safe or that the service will be available. \n",
"- **Use at your own risk** and keep backups of important data.\n",
"\n",
"---\n",
"\n",
"\n",
"### MinIO Access\n",
"Choose an identity provider and login:\n",
"\n",
"\n",
"\n",
"- **Buckets available:**\n",
" - `naa-vre-public` (read-only)\n",
" - `naa-vre-user-data` (read/write access to a folder with your username, e.g., `myname@gmail.com`)\n",
"\n",
"---\n",
"\n",
"\n",
"\n",
"### Create an Access Key\n",
"\n",
"1. Select **\"Access Keys\"**\n",
"2. Click **\"Create access key\"**\n",
"3. Click **\"Create\"**\n",
"4. **Save the keys for later!**\n",
"\n",
"---\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"## Use MinIO in NaaVRE\n",
"\n",
"1. Open your favorite VRE (see **NaaVRE Getting Started tutorial**)\n",
"2. Create a new notebook\n",
"\n",
"---\n",
"\n",
"\n",
"\n",
"## R Example\n",
"\n",
"```r\n",
"# Configuration (do not containerize this cell)\n",
"param_minio_endpoint = \"scruffy.lab.uvalight.net:9000\"\n",
"param_minio_region = \"nl-uvalight\"\n",
"param_minio_user_prefix = \"myname@gmail.com\" # Your personal folder in the naa-vre-user-data bucket\n",
"secret_minio_access_key = \"2ny7v9bdV6kVwh7n93SZ\"\n",
"secret_minio_secret_key = \"an1SP4eYLx4Awzn42lgVUmeHouPrgNp7cRcBLvrB\"\n",
"```\n",
"```r\n",
"# Access MinIO files\n",
"install.packages(\"aws.s3\")\n",
"library(\"aws.s3\")\n",
"\n",
"Sys.setenv(\n",
" \"AWS_S3_ENDPOINT\" = param_minio_endpoint,\n",
" \"AWS_DEFAULT_REGION\" = param_minio_region,\n",
" \"AWS_ACCESS_KEY_ID\" = secret_minio_access_key,\n",
" \"AWS_SECRET_ACCESS_KEY\" = secret_minio_secret_key\n",
")\n",
"\n",
"# List existing buckets\n",
"bucketlist()\n",
"\n",
"# List files in your personal folder in MinIO\n",
"get_bucket_df(bucket = \"naa-vre-user-data\", prefix = paste0(param_minio_user_prefix, \"/\"))\n",
"\n",
"# Upload file to MinIO\n",
"put_object(\n",
" bucket = \"naa-vre-user-data\",\n",
" file = \"myfile_local.csv\",\n",
" object = paste0(param_minio_user_prefix, \"/myfile.csv\")\n",
")\n",
"\n",
"# Download file from MinIO\n",
"save_object(\n",
" bucket = \"naa-vre-user-data\",\n",
" object = paste0(param_minio_user_prefix, \"/myfile.csv\"),\n",
" file = \"myfile_downloaded.csv\"\n",
")\n",
"\n",
"```\n",
"\n",
"## Python example\n",
"\n",
"```python\n",
"# Configuration (do not containerize this cell)\n",
"param_minio_endpoint = \"scruffy.lab.uvalight.net:9000\"\n",
"param_minio_user_prefix = \"myname@gmail.com\" # Your personal folder in the naa-vre-user-data bucket in MinIO\n",
"secret_minio_access_key = \"2ny7v9bdV6kVwh7n93SZ\"\n",
"secret_minio_secret_key = \"an1SP4eYLx4Awzn42lgVUmeHouPrgNp7cRcBLvrB\"\n",
"```\n",
"\n",
"```python\n",
"# Access MinIO files\n",
"from minio import Minio\n",
"mc = Minio(endpoint=param_minio_endpoint,\n",
" access_key=secret_minio_access_key,\n",
" secret_key=secret_minio_secret_key)\n",
"\n",
"# List existing buckets: get a list of all available buckets\n",
"mc.list_buckets()\n",
"\n",
"# List files in bucket: get a list of files in a given bucket. For bucket `naa-vre-user-data`, only list files in your personal folder\n",
"objects = mc.list_objects(\"naa-vre-user-data\", prefix=f\"{param_minio_user_prefix}/\")\n",
"for obj in objects:\n",
" print(obj.object_name)\n",
"\n",
"# Upload file to bucket: uploads `myfile_local.csv` to your personal folder on MinIO as `myfile.csv`\n",
"mc.fput_object(bucket_name=\"naa-vre-user-data\", file_path=\"myfile_local.csv\", object_name=f\"{param_minio_user_prefix}/myfile.csv\")\n",
"\n",
"# Download file from bucket: download `myfile.csv` from your personal folder on MinIO and save it locally as `myfile_downloaded.csv`\n",
"mc.fget_object(bucket_name=\"naa-vre-user-data\", object_name=f\"{param_minio_user_prefix}/myfile.csv\", file_path=\"myfile_downloaded.csv\")\n",
"```\n",
"\n",
"## License\n",
"\n",
"This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3b55e3c0-0edb-42a8-a26a-8dc1185e73a7",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python [conda env:biodt-hackathon25]",
"language": "python",
"name": "conda-env-biodt-hackathon25-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.11"
}
},
"nbformat": 4,
"nbformat_minor": 5
}