Github does not support expressions in the Docker URI, so we explicitly unroll the build matrix into a bunch of steps gated by a conditional expression instead. This works, but the downside is that Github builds all docker images in every matrix iteration, effectively building images n^2 times. All that without using any build cache. The result is that nightly builds easily spend more time in preparing the build images than actually building the packages.
This can be fixed by pre-building all necessary docker images in a separate workflow step, preferably using some sort of build cache (see actions/cached-docker-build-push). The build steps would then use pushed images.