Use pre-built images for nginx & service containers

[alxndrsn]

Related to #677

What's changed?

• docker-compose.yml now references service and nginx containers by specific version number
• github actions GHCR upload workflow has been added to the end of the Test workflow
• add a script for tagging releases and making appropriate updates to docker-compose.yml. If preferred, this could be converted to documentation

---

What has been done to verify that this works as intended?

• CI
• a lot of manual testing in a forked repo (https://github.com/alxndrsn/odk-central/actions)
• ran docker compose build locally, and watched the results:

...
[+] Building 4/4
 ✔ enketo      Built      0.0s
 ✔ postgres    Built      0.0s
 ✔ postgres14  Built      0.0s
 ✔ secrets     Built      0.0s

Why is this the best possible solution? Were any other approaches considered?

The obvious alternative was to keep the GHCR workflow in its own YAML file. It should then be possible to control the triggering of that workflow via "workflow dispatch" (https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#workflow_run). However, this approach is very hard to reason about and debug, and I did not get it working. The main benefit seems to be keeping code separate, but this doesn't seem worth the effort of getting it working. If modularising the build yml become a priority, alternative approaches can be reconsidered.

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

• should reduce the system requirements for performing upgrades
• might contain surprises for deployers who have made local changes to any of client, server, nginx etc.

Does this change require updates to documentation? If so, please file an issue here and include the link below.

I don't think so?

Before submitting this PR, please make sure you have:

• branched off and targeted the next branch OR only changed documentation/infrastructure (master is stable and used in production)
• verified that any code or assets from external sources are properly credited in comments or that everything is internally sourced

[lognaturel]

Nice! We can automate substituting in the tagged version in the docker compose file and then tag that commit. 👍

@matthew-white what do you think about running a script to release? The tag would then show up in Github and we could write the release notes there as usual.

I thought briefly about moving that functionality to the release Github action but that won't quite work -- we want to tag the commit that makes the change to the yml file.

I think we'll want to add some documentation on how to make code edits to frontend and backend since that will now require changes to the docker compose file as well. It's not something we highly recommend but it was easy in the past and we know some people do it.

We'll need to build from local source for the staging server. Maybe we could consider conditionally building or using an image based on a setting (could be a follow-up PR)?

[matthew-white]

I'd be OK with running a script. 👍 However, I don't understand yet the intended release process as a whole. Very possibly I'm just missing something. From the script, it looks like we'd run the script on the master branch of the central repo. But wouldn't we need to run the script on next before merging next into master? Otherwise, there would be a commit on the master branch that includes changes to the central repo without corresponding changes to central-backend and central-frontend. For example, updating Node requires changing central and sometimes also requires code changes to central-backend. If we merged that change to central into master and only then ran the script, then there'd be a commit and a short time window in which master doesn't contain working code. In other words, I'm wondering about the chance that a user could install Central at an awkward time between when the next branch is merged and the script is run. Is there a way to keep the release fully atomic? Could the script be run on the next branch instead of master? Or could the script merge next into master as part of its process?

[matthew-white]

Sort of a random thought / only somewhat related, but if we're going to have a script for the release, could one thing that the script does be generating version.txt? Like could we remove that generation from nginx.dockerfile? If so, then maybe we wouldn't need to include the .git/ directory in the Docker context.

[lognaturel]

could one thing that the script does be generating version.txt?

We also want to be able to use images for non-released builds so I think this is a possible idea but it would have to be part of the CI-based image building, not specifically part of the tagging/release script.

[lognaturel]

Latest thinking doc. @alxndrsn will revise this PR in light of those thoughts and describe how each end-user goal is met by the proposed approach.

[alxndrsn]

But wouldn't we need to run the script on next before merging next into master? Otherwise, there would be a commit on the master branch that includes changes to the central repo without corresponding changes to central-backend and central-frontend.

I think this is correct. To tackle this I have:

1. renamed release.sh to tag-release.sh, and
2. changed the branch check from only runs on master to cannot run on master

[alxndrsn]

I think we'll want to add some documentation on how to make code edits to frontend and backend since that will now require changes to the docker compose file as well. It's not something we highly recommend but it was easy in the past and we know some people do it.

Is there any risk of breaking these peoples' existing deployments? Would it be better for existing commands to continue building locally, and add new recommended commands which use pre-built images?

[lognaturel]

Is there any risk of breaking these peoples' existing deployments?

It would definitely be annoying for them. But if they're modifying source we know they have technical capacity and as long as we document a process I think it's ok.

Would it be better for existing commands to continue building locally, and add new recommended commands which use pre-built images?

Can you elaborate a bit on what that would look like? When you say commands, do you mean docker compose up, etc?

[alxndrsn]

When you say commands, do you mean docker compose up, etc?

Yes, so in this scenario the existing docker compose up etc. would still work for deployments building locally, and then documentation would change for new deployments to use pre-built images, perhaps with dedicated scripts e.g. ./restart-services.sh.

I don't think this necessarily simplifies things though, so I'm relieved you said this:

as long as we document a process I think it's ok.

[matthew-white]

@alxndrsn, just wanted to check whether there are any outstanding questions here. Are you blocked by anything? Once we get v2025.1 out the door, I'd be happy to meet/pair on this PR. I'm familiar with our release process(es) and how the different repos work, so I think I can add any context that's missing.