Skip to content

🐛 Fix storage for IBM COS (api_key) non-read ops. #1447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JosepSampe merged 3 commits into from
Oct 2, 2025
Merged

🐛 Fix storage for IBM COS (api_key) non-read ops. #1447

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
90ba185
🐛 Fix storage for IBM COS (api_key) non-read ops.
kikemolina3 Oct 1, 2025
da18535
Empty commit to add coauthors
kikemolina3 Oct 1, 2025
e2edd6e
🚨 Fix linter issue
kikemolina3 Oct 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config/config_template.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@
#ibm_cos:
#storage_bucket: <BUCKET_NAME>
#region: <REGION>
#service_instance_id: <SERVICE_INSTANCE_ID>
#endpoint: <REGION_ENDPOINT>
#private_endpoint: <PRIVATE_REGION_ENDPOINT>
#api_key: <API_KEY>
Expand Down
19 changes: 10 additions & 9 deletions docs/source/storage_config/ibm_cos.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,12 +127,13 @@ Alternative to using region, you can configure the public and private endpoints

### IBM Cloud Object Storage:

|Group|Key|Default|Mandatory|Additional info|
|---|---|---|---|---|
|ibm_cos | region | |yes | Region of your bucket. One of: `eu-gb`, `eu-de`, `us-south`, `us-east`, `br-sao`, `ca-tor`, `jp-tok`, `jp-osa`, `au-syd`. Lithops will use the region set under the `ibm` section if it is not set here|
|ibm_cos | api_key | |yes | API Key to your COS account. Not needed if using IAM API Key|
|ibm_cos | storage_bucket | | yes | The name of a bucket that exists in you account. This will be used by Lithops for intermediate data. You must provide HMAC Credentials if you want the bucket to be automatically created |
|ibm_cos | access_key_id | |no | HMAC Credentials. **Mandatory** if no api_key. Not needed if using IAM API Key|
|ibm_cos | secret_access_key | |no | HMAC Credentials. **Mandatory** if no api_key. Not needed if using IAM API Key|
|ibm_cos | endpoint | |no | Endpoint to your COS account. **Mandatory** if no region. Make sure to use the full path with 'https://' as prefix |
|ibm_cos | private_endpoint | |no | Private endpoint to your COS account. **Mandatory** if no region. Make sure to use the full path with 'https://' or http:// as prefix |
| Group | Key |Default|Mandatory| Additional info |
|---------|---------------------|---|---|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| ibm_cos | region | |yes | Region of your bucket. One of: `eu-gb`, `eu-de`, `us-south`, `us-east`, `br-sao`, `ca-tor`, `jp-tok`, `jp-osa`, `au-syd`. Lithops will use the region set under the `ibm` section if it is not set here |
| ibm_cos | api_key | |yes | API Key to your COS account. Not needed if using IAM API Key |
| ibm_cos | storage_bucket | | yes | The name of a bucket that exists in you account. This will be used by Lithops for intermediate data. You must provide HMAC Credentials if you want the bucket to be automatically created |
| ibm_cos | service_instance_id | |no | The service instance (CRN format) of your COS instance. **Mandatory** if no HMAC Credentials provided. | | | | |
| ibm_cos | access_key_id | |no | HMAC Credentials. **Mandatory** if no api_key. Not needed if using IAM API Key |
| ibm_cos | secret_access_key | |no | HMAC Credentials. **Mandatory** if no api_key. Not needed if using IAM API Key |
| ibm_cos | endpoint | |no | Endpoint to your COS account. **Mandatory** if no region. Make sure to use the full path with 'https://' as prefix |
| ibm_cos | private_endpoint | |no | Private endpoint to your COS account. **Mandatory** if no region. Make sure to use the full path with 'https://' or http:// as prefix |
8 changes: 7 additions & 1 deletion lithops/storage/backends/ibm_cos/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@ def load_config(config_data):
if 'private_endpoint' not in config_data['ibm_cos']:
raise Exception('You must provide the private_endpoint to access to IBM COS')
elif 'direct' not in config_data['ibm_cos']['private_endpoint']:
raise Exception('The private_endpoint you provided to access to IBM COS is not valid. You must use the "direct" endpoint')
raise Exception('The private_endpoint you provided to access to IBM COS is not valid. You must use the '
'"direct" endpoint')
if not config_data['ibm_cos']['private_endpoint'].startswith('http'):
raise Exception('IBM COS Private Endpoint must start with http:// or https://')

Expand All @@ -78,3 +79,8 @@ def load_config(config_data):
if 'region' not in config_data['ibm_cos']:
endpoint = config_data['ibm_cos']['endpoint']
config_data['ibm_cos']['region'] = endpoint.split('//')[1].split('.')[1]

if ('service_instance_id' not in config_data['ibm_cos']
and {'access_key_id', 'secret_access_key'}.isdisjoint(config_data['ibm_cos'])):
raise Exception('You must provide the service_instance_id or the (access_key_id,secret_access_key) to '
'access to IBM COS')
3 changes: 2 additions & 1 deletion lithops/storage/backends/ibm_cos/ibm_cos.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,8 @@ def __init__(self, ibm_cos_config):
aws_secret_access_key="",
aws_session_token=token,
config=client_config,
endpoint_url=service_endpoint
endpoint_url=service_endpoint,
ibm_service_instance_id=self.config.get('service_instance_id')
)

msg = STORAGE_CLI_MSG.format('IBM COS')
Expand Down