Allow the default docker command that packages gems to be overriden

[lwoodson]

Using env variable and include more debug information. This was necessary for us at onramp as we run our builds inside docker containers. When we started having native dependencies, this resulted in the serverless-ruby-package plugin attempting to run docker containers inside docker containers.

Docker in docker is a known and used operational pattern and can be enabled by mounting the docker socket from the host inside the container that will launch new containers. A side effect of this, however, is that mounts to the innermost container are not happening relative to its immediate parent container that spawned it. Rather, it needs to be against the filesystem on the outermost container host.

So the default command of:

docker run --rm \
           --volume "${localPath}:/var/task" \
           --entrypoint '/bin/bash' \
           ${dockerImage} \
           -c '/var/task/node_modules/serverless-ruby-package/build-gems.sh'

Works if you are running on a top level container on a local development machine, but NOT on a nested container. This allows the above command to be overriden with the SRP_DOCKER_COMMAND env variable as we are here in our particular project:

SRP_DOCKER_COMMAND=docker run --rm -v $$BUILDKITE_BUILD_DIRECTORY:/var/task --entrypoint /bin/bash amazon/aws-lambda-ruby:3.2 './node_modules/serverless-ruby-package/build-gems.sh'

Unit Tests

> yarn test
yarn run v1.22.4
$ jest
 PASS  __tests__/index.test.js
  ✓ captures the serverless configuration (1ms)
  ✓ captures the options
  ✓ hooks in before packaging deployment artifacts (1ms)
  ✓ hooks in before packaging an individual function
  ✓ disables 'Excluding development dependencies', which only applies to node projects (384ms)
  ✓ forces whitelisting files to package by excluding all files by default (348ms)
  ✓ include the bundler standalone files (336ms)
  ✓ preserve the includes specified in serverless configuration (342ms)
  ✓ include files for each gem needed by default bundler group - excluding .git/test files (398ms)

Test Suites: 1 passed, 1 total
Tests:       9 passed, 9 total
Snapshots:   0 total
Time:        2.399s, estimated 4s
Ran all test suites.
✨  Done in 3.40s.

Integration Tests

## BUILD PACKAGE FOR RUBY 3.2
ruby-package: Building gems with native extensions for linux
....
## VERIFY RUBY 3.2 FUNCTION CAN LOAD DEPENDENCIES
{:statusCode=>200, :body=>"{\"alpha\":\"first\",\"beta\":\"second\",\"redis_version\":\"5.0.6\",\"response\":\"71.211.6.165\n\"}"}
status_code: 200
## BUILD PACKAGE FOR RUBY 2.7
ruby-package: Building gems with native extensions for linux
....
## VERIFY RUBY 2.7 FUNCTION CAN LOAD DEPENDENCIES
{:statusCode=>200, :body=>"{\"alpha\":\"first\",\"beta\":\"second\",\"redis_version\":\"5.0.6\",\"response\":\"71.211.6.165\n\"}"}
status_code: 200
## CLEANING UP
Updated 1 path from the index

[joshuaflanagan]

Is there any way for the code to detect this situation and adapt for you? The proposed solution requires the user to have intimate details of the plugin's implementation details. By copying & pasting the current default command to make a couple tweaks, the user is "freezing" the command into their own codebase, and will miss out on any changes to the plugin.
Is it just the local path that needs to be overwritten?

[lwoodson]

A more targeted but less flexible change would be to allow ENV var to override the localPath here. In our case:

• build host: a CI worker node. Mount is something like /var/build/<project_checkout>:/app
• build container: the docker container launched by the host with all the dependencies for a serverless ruby project that has the project checkout mounted to /app and runs the serverless package command
• native extension container: the docker container spawned by serverless-ruby-package inside the build container. Here, localPath is evaluating to /app but since the build container is sharing the docker socket with the build host, localPath needs to be a path to where the project is on the build host. So the mount here should be /var/build/<project_checkout>:/var/task and not /app:/var/task

[lwoodson]

We may want to also include documentation for docker-in-docker in the readme. Many CI/CD systems will use a docker-in-docker approach.

[joshuaflanagan]

This seems like a reasonable approach, if we are unable to detect the "docker in docker" situation and determine the volume mount automatically.