Overhaul the Docker setup

[ldionne]

This patch rewrites the Docker setup used to create a deployable container running a production LNT server.

• Relocate all of the Docker-related files under docker/.
• Document the various Docker-related files.
• Use proper Docker secrets to transmit sensitive information to the LNT webserver entry point and the Postgres database.

With this setup, I am able to spin up a local LNT server instance with:

 docker compose --file docker/compose.yaml --env-file <secrets> up

An example of a secrets file would be

 LNT_DB_PASSWORD=foo
 LNT_AUTH_TOKEN=bar

[lukel97]

Can we avoid copying the entire source directory and only copy the required files to avoid clobbering the layers? This was originally done in 2d87edd, it helps keep image sizes small and reduces the cost of container registries over time etc.

[ldionne]

Ah, good point. Actually, I don't think we need anything except the installed Python package and the entrypoints, right? So we should be able to copy the sources during the build phase but not have them in the resulting image.

[ldionne]

Let me know what you think of the new approach with --mount=type=bind. I'm (obviously) learning Docker so I hope my usage is alright, but IIUC this should result in the smallest possible image.

[ldionne]

@lukel97 and I discussed this now at the LLVM Dev Meeting and basically we'd like to move the installation of dependencies into a separate step that runs before the installation of LNT itself. This allows caching the dependencies when only LNT itself changes.

This can be done by copying requirements.txt into the Docker image and getting rid of . in requirements.txt. Then, pip3 install -r requirements.txt will install only the dependencies, without requiring the actual sources in the Docker image.

This will be pursued in a separate PR.

[lukel97]

Should the client image be installing requirements.client.txt?

[ldionne]

This is actually creating an image that can run a server, I didn't think we wanted to produce an image that contains just the client, but I guess it makes some sense? In that case, I would suggest we produce three images:

• llvm-lnt-client
• llvm-lnt-server
• llvm-lnt-server-prod

The idea is basically that we produce base images that just contain (the various flavours of) LNT, and then on top of that we build an image that has the actual configuration to run in production (the docker entry point, etc).

Let me know if you think we should be building three images, or something else.

[ldionne]

@lukel97 I removed the usage of Docker secrets from this PR. However, we get the following warnings when building the Docker image:

 4 warnings found (use docker --debug to expand):
 - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "DB_PASSWORD") (line 40)
 - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "AUTH_TOKEN") (line 40)
 - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "DB_PASSWORD") (line 44)
 - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "AUTH_TOKEN") (line 45)

We'll need to figure out how to best use Docker Secrets while still providing a readily-usable Docker image in a separate PR.

[lukel97]

LGTM, thanks

[lukel97]

Can you land the postgres upgrade in a separate commit just in case the upgrade causes any issues and we want to revert?