Skip to content

Remove fog google blobstore support #5261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
kathap wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Remove fog google blobstore support #5261

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a87c93f
Remove fog google blobstore support
kathap Jul 3, 2026
8add09e
Remove fog_gcp_storage_options and Fog::Google dead code
kathap Jul 3, 2026
5418731
Remove fog_gcp_storage_options from config and test fixtures
kathap Jul 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ gem 'railties', '~> 8.1.1'

gem 'fog-aws'
gem 'fog-core', '~> 2.6.0'
gem 'fog-google', '~> 1.29.4'

gem 'cf-uaa-lib', '~> 4.0.9'

Expand Down
63 changes: 0 additions & 63 deletions Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,6 @@ GEM
debug (1.11.1)
irb (~> 1.10)
reline (>= 0.3.8)
declarative (0.0.20)
delayed_job (4.2.0)
activesupport (>= 3.0, < 9.0)
benchmark
Expand All @@ -85,8 +84,6 @@ GEM
logger
factory_bot (6.6.0)
activesupport (>= 6.1.0)
faraday (0.17.6)
multipart-post (>= 1.2, < 3)
ffi (1.17.4)
ffi (1.17.4-x86_64-darwin)
ffi (1.17.4-x86_64-linux-gnu)
Expand All @@ -103,19 +100,6 @@ GEM
excon (~> 1.0)
formatador (>= 0.2, < 2.0)
mime-types
fog-google (1.29.4)
addressable (>= 2.7.0)
fog-core (~> 2.5)
fog-json (~> 1.2)
fog-xml (~> 0.1.0)
google-apis-compute_v1 (~> 0.53)
google-apis-dns_v1 (~> 0.28)
google-apis-iamcredentials_v1 (~> 0.15)
google-apis-monitoring_v3 (~> 0.37)
google-apis-pubsub_v1 (~> 0.30)
google-apis-sqladmin_v1beta4 (~> 0.38)
google-apis-storage_v1 (>= 0.19, < 1)
google-cloud-env (>= 1.2, < 3.0)
fog-json (1.3.0)
fog-core
multi_json (~> 1.10)
Expand All @@ -124,30 +108,6 @@ GEM
nokogiri (>= 1.5.11, < 2.0.0)
formatador (1.2.3)
reline
google-apis-compute_v1 (0.86.0)
google-apis-core (>= 0.11.0, < 2.a)
google-apis-core (0.11.3)
addressable (~> 2.5, >= 2.5.1)
googleauth (>= 0.16.2, < 2.a)
httpclient (>= 2.8.1, < 3.a)
mini_mime (~> 1.0)
representable (~> 3.0)
retriable (>= 2.0, < 4.a)
rexml
google-apis-dns_v1 (0.36.0)
google-apis-core (>= 0.11.0, < 2.a)
google-apis-iamcredentials_v1 (0.17.0)
google-apis-core (>= 0.11.0, < 2.a)
google-apis-monitoring_v3 (0.54.0)
google-apis-core (>= 0.11.0, < 2.a)
google-apis-pubsub_v1 (0.45.0)
google-apis-core (>= 0.11.0, < 2.a)
google-apis-sqladmin_v1beta4 (0.61.0)
google-apis-core (>= 0.11.0, < 2.a)
google-apis-storage_v1 (0.32.0)
google-apis-core (>= 0.11.0, < 2.a)
google-cloud-env (1.6.0)
faraday (>= 0.17.3, < 3.0)
google-protobuf (4.35.1)
bigdecimal
rake (~> 13.3)
Expand All @@ -163,12 +123,6 @@ GEM
grpc (~> 1.41)
googleapis-common-protos-types (1.23.0)
google-protobuf (~> 4.26)
googleauth (1.8.1)
faraday (>= 0.17.3, < 3.a)
jwt (>= 1.4, < 3.0)
multi_json (~> 1.11)
os (>= 0.9, < 2.0)
signet (>= 0.16, < 2.a)
grpc (1.81.1)
google-protobuf (>= 3.25, < 5.0)
googleapis-common-protos-types (~> 1.0)
Expand Down Expand Up @@ -198,8 +152,6 @@ GEM
json-schema (6.2.0)
addressable (~> 2.8)
bigdecimal (>= 3.1, < 5)
jwt (2.10.3)
base64
kramdown (2.5.2)
rexml (>= 3.4.4)
kramdown-parser-gfm (1.1.0)
Expand All @@ -219,7 +171,6 @@ GEM
logger
mime-types-data (~> 3.2025, >= 3.2025.0507)
mime-types-data (3.2026.0414)
mini_mime (1.1.5)
mini_portile2 (2.8.9)
minitest (6.0.6)
drb (~> 2.0)
Expand All @@ -229,7 +180,6 @@ GEM
msgpack (1.8.3)
multi_json (1.21.1)
multipart-parser (0.1.1)
multipart-post (2.4.1)
mustache (1.1.2)
mustermann (3.1.1)
mutex_m (0.3.0)
Expand All @@ -252,7 +202,6 @@ GEM
ostruct (>= 0.2)
open3 (0.2.1)
openssl (4.0.2)
os (1.1.4)
ostruct (0.6.3)
parallel (2.1.0)
parallel_tests (5.7.0)
Expand Down Expand Up @@ -336,11 +285,6 @@ GEM
regexp_parser (2.12.0)
reline (0.6.3)
io-console (~> 0.5)
representable (3.2.0)
declarative (< 0.1.0)
trailblazer-option (>= 0.1.1, < 0.2.0)
uber (< 0.2.0)
retriable (3.8.0)
reverse_markdown (3.0.2)
nokogiri
rexml (3.4.4)
Expand Down Expand Up @@ -435,10 +379,6 @@ GEM
pg (>= 0.18.0, != 1.2.0)
sequel (>= 4.38.0)
sexp_processor (4.17.5)
signet (0.22.0)
addressable (~> 2.8)
faraday (>= 0.17.5, < 3.a)
jwt (>= 1.5, < 4.0)
simplecov (0.13.0)
docile (~> 1.1.0)
json (>= 1.8, < 3)
Expand Down Expand Up @@ -501,11 +441,9 @@ GEM
thor (1.5.0)
tilt (2.7.0)
timecop (0.9.11)
trailblazer-option (0.1.2)
tsort (0.2.0)
tzinfo (2.0.6)
concurrent-ruby (~> 1.0)
uber (0.1.0)
unicode-display_width (3.2.0)
unicode-emoji (~> 4.1)
unicode-emoji (4.2.0)
Expand Down Expand Up @@ -547,7 +485,6 @@ DEPENDENCIES
fluent-logger
fog-aws
fog-core (~> 2.6.0)
fog-google (~> 1.29.4)
googleapis-common-protos (>= 1.8.0)
hashdiff
httpclient
Expand Down
4 changes: 0 additions & 4 deletions config/cloud_controller.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,6 @@ resource_pool:
blobstore_type: local-temp-storage
fog_connection: {}
fog_aws_storage_options: {}
fog_gcp_storage_options: {}

packages:
app_package_directory_key: "cc-packages"
Expand All @@ -193,22 +192,19 @@ packages:
blobstore_type: local-temp-storage
fog_connection: {}
fog_aws_storage_options: {}
fog_gcp_storage_options: {}

droplets:
droplet_directory_key: cc-droplets
max_staged_droplets_stored: 42
blobstore_type: local-temp-storage
fog_connection: {}
fog_aws_storage_options: {}
fog_gcp_storage_options: {}

buildpacks:
buildpack_directory_key: cc-buildpacks
blobstore_type: local-temp-storage
fog_connection: {}
fog_aws_storage_options: {}
fog_gcp_storage_options: {}

db_encryption_key: "asdfasdfasdf"
database_encryption:
Expand Down
7 changes: 2 additions & 5 deletions lib/cloud_controller/blobstore/client_provider.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
require 'cloud_controller/blobstore/local/local_client'
require 'cloud_controller/blobstore/safe_delete_client'
require 'cloud_controller/blobstore/storage_cli/storage_cli_client'
require 'google/apis/errors'

module CloudController
module Blobstore
Expand Down Expand Up @@ -40,8 +39,7 @@ def provide_fog(options, directory_key, root_dir)
root_dir: root_dir,
min_size: options[:minimum_size],
max_size: options[:maximum_size],
aws_storage_options: options[:fog_aws_storage_options],
gcp_storage_options: options[:fog_gcp_storage_options]
aws_storage_options: options[:fog_aws_storage_options]
)

logger = Steno.logger('cc.blobstore')
Expand All @@ -52,8 +50,7 @@ def provide_fog(options, directory_key, root_dir)
# and https://github.com/fog/fog-aws/issues/265
# and intermittent GCS blobstore download errors
errors = [Excon::Errors::BadRequest, Excon::Errors::SocketError, SystemCallError,
Excon::Errors::InternalServerError, Excon::Errors::ServiceUnavailable,
Google::Apis::ServerError, Google::Apis::TransmissionError, OpenSSL::OpenSSLError]
Excon::Errors::InternalServerError, Excon::Errors::ServiceUnavailable, OpenSSL::OpenSSLError]
retryable_client = RetryableClient.new(client:, errors:, logger:)

Client.new(ErrorHandlingClient.new(SafeDeleteClient.new(retryable_client, root_dir)))
Expand Down
9 changes: 1 addition & 8 deletions lib/cloud_controller/blobstore/fog/fog_client.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,14 @@ def initialize(connection_config:,
root_dir: nil,
min_size: nil,
max_size: nil,
aws_storage_options: nil,
gcp_storage_options: nil)
aws_storage_options: nil)
@root_dir = root_dir
@connection_config = connection_config
@directory_key = directory_key
@cdn = cdn
@min_size = min_size || 0
@max_size = max_size
@aws_storage_options = aws_storage_options
@gcp_storage_options = gcp_storage_options
logger.warn('blobstore.fog-deprecated', message: DEPRECATION_MESSAGE)
end

Expand Down Expand Up @@ -142,11 +140,6 @@ def formatted_storage_options
opts['x-amz-server-side-encryption'] = encrypt_opt
opts

elsif [Fog::Google::StorageJSON, Fog::Google::StorageXML].include?(connection.service)
return {} unless @gcp_storage_options

@gcp_storage_options

else
{}

Expand Down
4 changes: 0 additions & 4 deletions lib/cloud_controller/blobstore/fog/providers.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,8 @@
require 'fog/core'

# This can be removed once fog-google fixes
# https://github.com/fog/fog-google/issues/421, which will allow us to upgrade
# fog-core to 2.2.4.
original = Fog::Logger[:deprecation]
Fog::Logger[:deprecation] = nil

require 'fog/aws'
require 'fog/google'

Fog::Logger[:deprecation] = original
1 change: 0 additions & 1 deletion lib/cloud_controller/blobstore/storage_cli/storage_cli_client.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ class StorageCliClient < BaseClient
# DEPRECATED: Legacy fog provider names (remove after migration window)
LEGACY_PROVIDER_TO_STORAGE_CLI_TYPE = {
'AWS' => 's3',
'Google' => 'gcs',
'webdav' => 'dav'
}.freeze

Expand Down
4 changes: 0 additions & 4 deletions lib/cloud_controller/config_schemas/api_schema.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,6 @@ class ApiSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand All @@ -228,7 +227,6 @@ class ApiSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand All @@ -242,7 +240,6 @@ class ApiSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand All @@ -255,7 +252,6 @@ class ApiSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand Down
1 change: 0 additions & 1 deletion lib/cloud_controller/config_schemas/blobstore_benchmarks_schema.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ class BlobstoreBenchmarksSchema < VCAP::Config
optional(:cdn) => Hash,

fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,

optional(:resource_directory_key) => String,
optional(:buildpack_directory_key) => String,
Expand Down
4 changes: 0 additions & 4 deletions lib/cloud_controller/config_schemas/clock_schema.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,6 @@ class ClockSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand All @@ -143,7 +142,6 @@ class ClockSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand All @@ -156,7 +154,6 @@ class ClockSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand All @@ -168,7 +165,6 @@ class ClockSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand Down
4 changes: 0 additions & 4 deletions lib/cloud_controller/config_schemas/deployment_updater_schema.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,6 @@ class DeploymentUpdaterSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand All @@ -138,7 +137,6 @@ class DeploymentUpdaterSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand All @@ -151,7 +149,6 @@ class DeploymentUpdaterSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand All @@ -163,7 +160,6 @@ class DeploymentUpdaterSchema < VCAP::Config
fog_connection: Hash,
optional(:connection_config) => Hash,
fog_aws_storage_options: Hash,
fog_gcp_storage_options: Hash,
optional(:webdav_config) => Hash,
optional(:cdn) => Hash
},
Expand Down
Loading
Loading