Enable Filebeat's Journald input in all docker image variants #7995
Conversation
Filebeat's Journald input requires `journalctl`, so the systemd package is added to all docker image variants.
|
This pull request does not have a backport label. Could you fix it @belimawr? 🙏
|
belimawr
added
Team:Elastic-Agent-Data-Plane
…wr/elastic-agent into 44040-journalctl-on-docker-iamges
|
💚 Build Succeeded
History
cc @belimawr |
|
Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane) |
pierrehilbert
added
the
Team:Elastic-Agent-Control-Plane
|
Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane) |
|
@belimawr please add the image size comparison to the description. |
| @@ -73,15 +73,15 @@ ENV BEAT_SETUID_AS={{ .user }} | |||
| {{- if contains .from "ubi-minimal" }} | |||
| RUN for iter in {1..10}; do \ | |||
| microdnf update -y && \ | |||
| microdnf install -y tar gzip findutils shadow-utils ca-certificates gawk libcap xz && \ | |||
| microdnf install -y tar gzip findutils shadow-utils ca-certificates gawk libcap xz systemd && \ | |||
There was a problem hiding this comment.
How much bigger does this make the images?
Also, we don't want it in every container, it would be in elastic-agent, elastic-agent-complete but not elastic-agent-slim. This is really dependent on how much the size increase is as we do not want them getting bigger since they are already huge.
There was a problem hiding this comment.
The downside of using the elastic-agent-complete image is that people who don't use synthetics but want to read node logs on K8S now have to deploy a bunch of browser dependencies and such.
Perhaps this would be best addressed with an elastic-agent-journald container. We already have a lot of containers though so I'm not sure this clarifies anything with respect to which one someone should use.
| microdnf clean all && \ | ||
| exit_code=0 && break || exit_code=$? && echo "microdnf error: retry $iter in 10s" && sleep 10; done; (exit $exit_code) | ||
| {{- end }} | ||
|
|
||
| {{- if contains .from "wolfi" }} | ||
| RUN for iter in {1..10}; do \ | ||
| apk fix && \ | ||
| apk add --no-cache ca-certificates curl gawk shadow bash && \ | ||
| apk add --no-cache ca-certificates curl gawk shadow bash systemd && \ |
There was a problem hiding this comment.
We shouldn't include systemd in wolfi because it is going to significantly broaden it's exposure to CVEs, the point of Wolfi is to minimize the OS level CVEs we are exposed to. Same rationale as Filebeat: https://github.com/elastic/beats/pull/44056/files#r2064612269
|
So I went ahead and found my notes from when we did the initial switch from ubuntu-based to ubi-based images, and I remembered the quirks with journalctl and systemd package. To this end, I built all images with this PR and this is the comparison the images from main, in terms of size: arm64: amd64: Just from the above comparisons, and the security-based reasons, I would say that wolfi-based images should not feature systemd. Now for the ubi-based ones, if the on average 50MB of an increase is prohibitive, we could try out having an intermediate step that installs systemd package and copies the bare minimum of journalctl dependencies to the final image I have to test the above out but AFAIK PS: I would also propose introducing a k8s integration test that tests the journald input 🙂 |
|
IMO based on the above we keep systemd out of the slim, service, and wolfi variants. We would add systemd/journald to the plain |
|
(service has no need for journald/systemd it's the agentless container). |
What does this PR do?
Filebeat's Journald input requires
journalctl, so the systemd package is added to all docker image variants.Why is it important?
Currently it is only possible to use integrations that requires the journald input with the complete docker image variant, this PR allows it to be used with any docker image variant. The complete docker image is the largest with more than 3Gb, while the standard Elastic-Agent image is less than 2Gb.
Checklist
I have commented my code, particularly in hard-to-understand areasI have made corresponding change to the default configuration filesI have added tests that prove my fix is effective or that my feature works./changelog/fragmentsusing the changelog toolI have added an integration test or an E2E test## Disruptive User ImpactHow to test this PR locally
journalctl:Related issues
journalctlto Filebeat and Elastic-Agent docker images beats#44040Questions to ask yourself