Feature Request: Container updating (auto and manual)

[STaRDoGG]

Is your feature request related to a problem? Please describe.
Currently there is no container updating functionality within Portainer, so I use a WatchTower container to keep my containers up to date instead. It's good, but also a little bit quirky to use, i.e. for manually updating a single container.

Describe the solution you'd like
It would be great if Portainer has similar functionality as WatchTower for updating any containers being managed. This could look like:

• Auto-check for any container updates, and if found, update using the same arguments as the original
• Ability to manually check for/update a container just by clicking an "update check" icon in the "Quick Actions" list of icons.
• A button near the top that does the same if you want to put a check in the checkboxes for several individual containers.

Additional context:
Related to #3197

[STaRDoGG]

Thanks @itsconquest , not a total dupe (as it looks like the other thread was meant to just keep Portainer itself updated, and this request is about keeping all containers updated) but close enough. Appreciate it!

[ghost]

@STaRDoGG It wasn't quite clear enough after reading the description, so I have updated the description and re-opened this issue :) Indeed the two feature requests are related, but in fact refer to different functionality in Portainer.

[cpcodes]

I have desired this type of functionality for awhile. It boggles my mind that there isn't even a stand alone tool that does this. Tools such as Watchtower and Ouroboros auto-update containers, which often causes problems when major functionality changes exist in new images.
Unraid has a Docker management interface that is close to ideal, but it is not available separately. Portainer will update the container for you if you rebuild, but does not provide a means to check if there is a newer version of the image - you must rebuild blind, or manually go to the Docker Hub page and see if it is updated. Also, rebuild is not an option from the container list - you have to view container details for each container one at a time to update multiple containers.
In short, in the container list allow columns that show current and latest version of the image the container is based on and add a rebuild button to the buttons at the top would be a minimum.
Additional features, such as highlighting the current image version field when it is less than the latest image version, hyperlink from the latest image version to the Docker hub (or whatever repository) change-log for that version (and/or a summary or first few lines of the change-log on hover), and some roll-back capability would all be welcome, but are just icing.
The latest version info would also be available in the image list, of course, since that is where it is primarily attached to. The schedule for updating this info (checking for newer versions from repositories) could also be customizable just as any other scheduled task in Portainer, likely as an interval (such as snapshots) or as a cron style option, or anything in between.

[chmac]

I would really like this feature, but most importantly, the option to "check for updates". If portainer would show me which containers had updates available, then I could manually click through them all to do the updates. But as @cpcodes said, there's no real alternative currently, except to rebuild all the containers one by one just in case they have updates...

[Manfred73]

Automatic update from within Portainer would be great.

[SvenMazet]

^^ this! should be an option!

[arraylabs]

This would be great.

[pkellner]

Be nice to have. Especially since portainer is already doing the registry login. I'm currently struggling with getting portainer, watchtower and ghcr.io to work together with private containers.

[zoe-tek]

After searching i found this and would like this feature as well.

[Hubro]

This feature was the only reason I just installed Portainer 😄

I'm tired of keeping my Docker containers up to date manually, so I assumed a management GUI would be able to do it for me. Pretty perplexed Portainer doesn't have this functionality.

[Manfred73]

Have a look at Watchtower docker image. It also works really well for automatically updating your docker containers. It's not hard to install and configure, you can just install the docker watchtower image.

[GlassedSilver]

Have a look at Watchtower docker image. It also works really well for automatically updating your docker containers. It's not hard to install and configure, you can just install the docker watchtower image.

Does Watchtower respect starting containers in the right order like Portainer does though?

I know, I know... "depends on" is not supposed to be used like that and race conditions aren't taken care of, still better to use that over nothing in a hobby environment over nothing.

I mean I am using Watchtower with my Portainer instance, and it seems to work, but it'd be nice if this was a thing in Portainer.

[alexander-potemkin]

@Manfred73 , thank you. Could it all be done from within Portainer itself?
The reason I'm asking, it's because I'm looking for an infrastructure automation on top of Portainer.

[Manfred73]

Have a look at Watchtower docker image. It also works really well for automatically updating your docker containers. It's not hard to install and configure, you can just install the docker watchtower image.

Does Watchtower respect starting containers in the right order like Portainer does though?

I know, I know... "depends on" is not supposed to be used like that and race conditions aren't taken care of, still better to use that over nothing in a hobby environment over nothing.

I mean I am using Watchtower with my Portainer instance, and it seems to work, but it'd be nice if this was a thing in Portainer.

I think it does: https://containrrr.dev/watchtower/linked-containers/

[Manfred73]

@Manfred73 , thank you. Could it all be done from within Portainer itself? The reason I'm asking, it's because I'm looking for an infrastructure automation on top of Portainer.

Not sure if Portainer supports this already, that's why I started using Watchtower for this as some others suggested.

[jeichenseerNRX]

The Watchtower links on this page got hijacked, and now point to a shopping site. Just FYI....

[lps-rocks]

I'm finding that using portainer combined with watchtower causes unnecessary restarts to containers when they're created via a stack in portainer, watchtower updates them, and then i update the stack configuration from portainer. Any containers that were updated by watchtower since the last stack configuration change, but not modified in the current stack configuration, get recreated by portainer.

[lucasgcbkhomp]

So, this golden feature has totally been 4 years in the backburner.

In the same line, the more ignorant suggestion for this userbase is to straight up make an interface within Portainer to manage Watchtower and make the necessary changes for it to behave alongside it. Perhaps that's more realistic.

[Manfred73]

I'm hardly using the Portainer interface for managing containers. Always use scripts and docker compose from the command line for that.

[alexander-potemkin]

@Manfred73 , for the sake of the interest: what use Portainer for then? As an API provider?

[lps-rocks]

@Manfred73 , for the sake of the interest: what use Portainer for then? As an API provider?

Please stay on topic. If you want to discuss this, please do it out of band.

I'm hardly using the Portainer interface for managing containers. Always use scripts and docker compose from the command line for that.

Please stay on the topic of issues surrounding auto-updating. You saying you're not using portainer for anything does not help and adds nothing to the conversation.

[alexander-potemkin]

@lps-rocks could you enable sub-threads, please? There is nothing wrong/bad about your clients learning new ways to use your product.

[lucasgcbkhomp]

@lps-rocks could you enable sub-threads, please? There is nothing wrong/bad about your clients learning new ways to use your product.

It's not their product, they're a user and an issue participant as they've informed the limitation of Watchtower earlier. They receive messages because they are subscribed and so do I and possibly everyone else here. The irony is that attempts to "backseat moderation" for off-topic simply leads to further off-topic discussion, like this.

@Manfred73 , for the sake of the interest: what use Portainer for then? As an API provider?

Since we are on the topic of the off topic, I believe Portainer has little use for someone who does understand Docker. It's most useful for introducing Docker to a team that does not understand it but could benefit from the ephemerality.

One useful thing is that instead of giving full access of every container to everyone in the team by providing a sudoer account, you may use Portainer's role based access control system as an interface to keep the developers in check and create their own services without endangering others by combining this with a registry. I use mine for developers who are learning docker to leave test applications running, the interface has been assisting them in understanding core concepts within the VM I've set up for them, they can easily access logs and set up the policies without spaghetting in the CLI and lambasting the existence of core services such as watchtower that they should not touch.

But again, this ultimately feels like a tool for simple things (hence, this ultimately golden feature has been unprioritized for years). For a more robust, production ready, and resource friendly approach, employing Kubernetes clusters with ArgoCD seems to be the superior GitOps approach.

[belthesar]

I'm interested in this feature from a few perspectives:

For workloads that are appliance-style containers, having something that performs an auto-update, assuming a well tagged version pin to prevent what should be breaking changes would be quite beneficial. This has been outlined quite well. However, I also would appreciate this from a security concern perspective.

I do not want to expose additional services to the Docker socket on my Docker hosts. While watchtower has been cited as an alternative that can perform this, that is yet another service that needs exposure to the Docker socket that, which has a significant amount of power (see Docker daemon attack surface, Don't expose the Docker socket (not even to a container), etc. I leverage Traefik on my Docker hosts to provide information about my containers, and to protect my infrastructure from a highly exposed attack surface, I couple that with Tecnavita's docker socket proxy to limit Traefik's exposure to a read-only interface with only the information it needs.

Portainer is already treated as a privileged (not in the Capital P Privileged, from the Docker, but the security concept) container, with its agents receiving access to the Docker socket. I do not feel it wise to perform this update. Portainer also already checks container image state, and can detect when updates are available based on the tag you have built the container with. The only additional information it requires to be able to perform this task is a timeframe to perform automatic updates, and a way to flag containers to opt in/out of the scheduled updates.

As it stands now, in order to get this functionality, I'm in the process of writing a tool that uses the Swagger-generated API client to be able to iterate through the containers in an environment, detect if they need updates, compare that with a list of containers I wish to update, and to perform the updates via cron. I could do this by creating webhooks for each container I want to auto-update, and call those individually, but in some senses, I'm writing this as a proof of concept for how the update function could work using Portainer's own data, and even then, I wouldn't be able to pull the data on whether a container needed updating by triggering a webhook, leading to needless service restarts.

As a part of the CI pipeline, the webhook feature is more than sufficient, and works great. However, for cases where these appliance-style workloads are being run, which could be all sort of back office or general business workloads, having Portainer be able to automatically manage the updates for these systems would be a great boon to the operators maintaining those systems.

[jamescarppe]

Have you had a look at our automatic update functionality for Git-deployed stacks? It's a BE-only feature, but this alongside related features like change window support might tick a lot of the boxes you're referring to here. This knowledge base article describes how it works.

[belthesar]

Thanks for replying @jamescarppe, and for the resources. Unfortunately, this does not appear to solve the issue. Based on my understanding (and please, clarify if I'm missing something here), the update mechanism is triggered when Portainer detects that the git commit hash changes. This would be beneficial for stack updates when the image tag is incremented, and I can see that being quite beneficial in cases to provide GitOps, similar to how ArgoCD works on the k8s world. This unfortunately does not solve any of my workflow scenarios, because whether I'm interacting with the Portainer UI, or I'm making a "bump commit" to the git repo to perform this work, I still need to sense whether the container image has an update, and this is data that Portainer already collects (as indicated by the image indicator light icon next to its name), and then ultimately perform an action to handle this. Let me provide a concrete example of a use case in which this would be beneficial to me, and I believe, many other users:

I currently use Portainer to manage two Docker Standalone instances in my home lab, although I had this same struggle at my previous job where I managed a handful of Docker Standalone instances. While I do have containers and stacks that I have deployed that I manage for applications that I have written (in which a GitOps workflow is indeed quite handy to assist with deployment), in both my personal and professional scenarios, I also have a fair number of applications that I've deployed as appliances. While I can cite a number of different services I've run both in my home lab and for my employer like HR tools for time tracking, a self-hosted GitLab instance, monitoring and management tools, we'll use Traefik, which I use for ingress.

Traefik uses semantic versioning, and makes a commitment to not make breaking API and configuration changes between patch versions. As a result, rather than pinning my image to a specific patch release, I pin to minor version (as of right now, v2.10). Currently, whether the container is in a stack, or is standalone, in order to redeploy, I'm required to sense that there is a new version of the image, and then execute the redeployment process to build a new container with the new image version. My current process to do this is:

1. Log into the Portainer UI
2. See if any of my workloads have red dots
3. For each container, use the UI to redeploy the container with the latest image (which respects my use of image tags to pull the latest version of that container that should have no compatibility issues)

I already have the controls available by virtue of the image vendor using SemVar to give me reasonable confidence that automatically patching will not cause breaking issues (and even if it did, the ownership of that is on me.) All that remains is for there to be a mechanism to trigger those updates.

With the constraints and facts below:

1. Giving access to the Docker socket on a Docker host should not be done lightly. The Portainer Agent running on each host in the environment already creates a sufficiently secure control plane for managing hosts, and I do not wish to introduce additional services with access to such a sensitive resource to limit surface area exposure to sensitive resources.
2. I only want to perform container updates when a new image is available using the tag filters I have configured for the containers and stacks running on my Portainer-managed hosts.
3. Portainer already collects the necessary information about container images (at least, from Docker Hub hosted images) to determine whether a new image for the given specified tag exists.
4. Portainer already has facilities for scheduling change windows to allow operators to ensure that container updates happen at the least impactful time for their environment
5. Portainer already has methods for performing a container rebuild with latest image pull both via the UI and via Webhook

it stands to reason that the Portainer platform is an ideal tool to support for automatic container rebuilds when new images for the container's image tag filter are ready.

[jamescarppe]

Thanks for the detailed reply! You're correct, the automatic updates do rely on the commit hash, and were indeed designed around filling the GitOps niche.

We've strayed away from doing automatic container updates as you describe in the past for a few reasons, including that Watchtower already does it and people are mostly fine with using that, but I do agree that you would want to reduce the amount of systems with socket access where possible. We've also been cautious about it as in most production deployments you would only really want to upgrade image versions manually in order to avoid taking things offline or introduce unknown issues from new versions, but your feedback re semantic versioning is helpful for a real-world use case of how you might do this in production.

I'll pass all your feedback here on to the team - the more detail we can get from real users as to how they would use the feature, what problems it would solve, etc, the more we can understand and prioritize the feature for future development.

[alexander-potemkin]

@jamescarppe , you might want to check Cloudron.io approach for that reason. The whole system is built on more or less automatic updates.

[Taztic]

I fully agree with @belthesar's comments. From a professional, security point of view, Watchtower is not an alternative and
from a private point of view, I would also prefer a one-hand solution.
I would also like to draw your attention to the almost 200 👍 for @STaRDoGG post from 2019.