Automate performance benchmarking

[mxinden]

Overarching tracking issue: #63

#163 allows running the various perf protocol implementations (libp2p/specs#478) on cloud instances.

Ideally we have a CI setup, that allows users to trigger the above. This removes the need for users to run it on their local machine.

Workflow

Starting point: libp2p implementation maintainer wants to test a new release candidate.

3. They open a pull request to libp2p/test-plans, adding their release candidate to perf/impl.
4. They trigger the automation via a GitHub comment.
5. CI runs:
   1. Generate an ssh keypair and set the public key at perf/terraform/user.pub.
   2. Run cd perf/terraform && terraform apply.
   3. Run cd perf/runner && npm run start -- --client-public-ip $(terraform output -raw -state ../terraform/terraform.tfstate client_public_ip) --server-public-ip $(terraform output -raw -state ../terraform/terraform.tfstate server_public_ip). This will write the benchmark results to perf/runner/benchmark-results.json.
   4. Run cd perf/terraform && terraform destroy.
   5. Push perf/runner/benchmark-results.json to the pull request.
6. Depending on the benchmark results:
   • In case the automation catches a regression, the maintainer can cut another patch release and update the libp2p/test-plans pull request and retrigger the automation.
   • In case the results are fine, the maintainer can cut the release. Once released, they update the libp2p/test-plans pull request, retrigger the automation and merge. Thus libp2p/testplans master branch always contains the latest benchmarking results.

[mxinden]

//CC @galargh I extracted the automation section from #163 to this new issue.

[galargh]

Thanks :) Here's the first pass at the automation - mxinden#2

[mxinden]

First successful automation run on libp2p/test-plans happened today. See #63 (comment). Thus merging here.

Thank you @galargh!