Optimize build times for Docker images

[bartjkdp]

Currently a build takes around 22 minutes. Investigate why.

[FayKn]

Issue ended up being multi arch builds, #36 resolves this with matrix builds on x86 and arm runners

build takes <4 minutes now.