Enhance flexibility, add GitLeaks, focus on Dockerfiles only

README.md

@@ -1,7 +1,7 @@
 # X9Containers
-Docker containers to finger pointing pesky breaches from your running images.
+Collection of Docker containers to finger pointing pesky breaches from your running images.
 
-It's nothing really new: X9Containers use existing tools to find malwares and vulnerabilities using [Trivy](https://github.com/aquasecurity/trivy) and [ClamAV](https://github.com/Cisco-Talos/clamav) in a [multi-stage building approach](https://docs.docker.com/develop/develop-images/multistage-build/), copying the whole filesystem to be analyzed at each stage - at `distroless.` suffix.
+It's nothing really new: X9Containers use existing tools to find malwares, vulnerabilities and secrets using [Trivy](https://github.com/aquasecurity/trivy), [ClamAV](https://github.com/Cisco-Talos/clamav) and [GitLeaks](https://github.com/zricethezav/gitleaks) in a [multi-stage building approach](https://docs.docker.com/develop/develop-images/multistage-build/), copying the whole filesystem to be analyzed at each stage - at `distroless.` suffix.
 
 But the analysis can also occur during building, so it can run into the intermediate container, where we hope to find things dynamically - at non `distroless.` suffix.
 
@@ -13,48 +13,84 @@ Said that...
 
 ### Usage
 
-Intended to just print each vulnerability scanner output to the standard output:
+This snippet is intended to just print each report to the standard output, so grab a drink and figure out a best use case based on your needs:
 
 ```sh
 # You need to choose the appropriate Dockerfile which will run on top of the target image container
-curl https://raw.githubusercontent.com/olxbr/X9Containers/main/debian.clamav.trivy.X9.Dockerfile --output X9.Dockerfile
-
-# Where IMAGE=${TARGET_IMAGE}:${VERSION} is the target image for scanning
-# and TRIVY_SEVERITY=${TRIVY_SEVERITY} is a Trivy comma separated threat levels to consider
-docker build -f X9.Dockerfile -t suspectimage --build-arg IMAGE=${TARGET_IMAGE}:${VERSION} --build-arg TRIVY_SEVERITY=${TRIVY_SEVERITY} --quiet .
+curl https://raw.githubusercontent.com/olxbr/X9Containers/main/distroless.clamav.trivy.gitleaks.X9.Dockerfile --output X9.Dockerfile
+
+# ARGS are for:
+# - REGISTRY is the exactly registry that contains all images used in this analysis
+# - *_IMAGE are Docker images in said registry
+# - TARGET_IMAGE it is a local build image chosen for analysis
+# - TRIVY_SEVERITY is a Trivy comma separated threat level. See https://aquasecurity.github.io/trivy/v0.20.0/vulnerability/examples/filter/
+# - WKDIR is the workdir used to held all your application stuff
+docker build -f X9.Dockerfile -t suspectimage \
+  --build-arg REGISTRY=${REGISTRY} \                                        # docker.io
+  --build-arg CLAMAV_IMAGE=${CLAMAV_IMAGE}:${CLAMAV_IMAGE_VERSION} \        # clamav/clamav:latest
+  --build-arg TRIVY_IMAGE=${TRIVY_IMAGE}:${TRIVY_IMAGE_VERSION} \           # aquasec/trivy:latest
+  --build-arg GITLEAKS_IMAGE=${GITLEAKS_IMAGE}:${GITLEAKS_IMAGE_VERSION} \  # zricethezav/gitleaks:latest
+  --build-arg BASE_IMAGE=${ALPINE_IMAGE}:${ALPINE_IMAGE_VERSION} \          # alpine:latest
+  --build-arg TARGET_IMAGE=${TARGET_IMAGE}:${TARGET_IMAGE_VERSION} \        # my/super/suspect/docker/local/builded/image:latest
+  --build-arg TRIVY_SEVERITY=${TRIVY_SEVERITY} \                            # CRITICAL
+  --build-arg WKDIR=${TRIVY_SEVERITY} \                                     # app
+  --quiet \
+  .
 docker create --name suspectcontainer suspectimage
 docker cp suspectcontainer:/scans ./scans
 
-# Do whatever you need with the artifacts, in this case it will only be printed in console:
+# Do whatever you need with the reports inside scans directory and do your own post execution cleanup:
 for i in scans/* ; do \
   cat $i ; \
-  echo "********** END OF $i ********** ; \
+  printf "\n\n********** END OF $i **********\n\n" ; \
 done
 ```
 
-#### Usage with GitHub Actions
-
-If you feel lazy and just want to blow up some pipeline across GitHub neighborhood, read below.
+Sample output
 
-First, build your Docker image as usual inside your pipeline:
 ```
-- name: Check out repository
-  uses: actions/checkout@v2
+repo,line,commit,offender,leakURL,rule,tags,commitMsg,author,email,file,date
+,REDACTED,,REDACTED,,AWS Access Key,"key, AWS",,,,aaaa.txt,0001-01-01T00:00:00Z
 
-...
 
-- name: Build Docker Image
-  run: docker build -t myleetimage .
-```
+********** END OF scans/gitleaks-leaks-result.txt **********
 
-And then give a try with Actions:
-```
-- name: Perform X9Containers Scan
-  uses: olxbr/X9Containers@main
-  timeout-minutes: 6
-  with:
-    image: myleetimage:latest
-    distro: distroless.clamav.trivy
-    trivy_severity: CRITICAL
-    ignore_threats: false
+2021-10-11T01:12:21.287Z	INFO	Need to update DB
+2021-10-11T01:12:21.287Z	INFO	Downloading DB...
+2021-10-11T01:12:24.437Z	INFO	Detecting Debian vulnerabilities...
+
+debuerreotype (debian 11.0)
+===========================
+Total: 2 (CRITICAL: 2)
+
++-----------+------------------+----------+-------------------+------------------+--------------------------------------+
+|  LIBRARY  | VULNERABILITY ID | SEVERITY | INSTALLED VERSION |  FIXED VERSION   |                TITLE                 |
++-----------+------------------+----------+-------------------+------------------+--------------------------------------+
+| libssl1.1 | CVE-2021-3711    | CRITICAL | 1.1.1k-1          | 1.1.1k-1+deb11u1 | openssl: SM2 Decryption              |
+|           |                  |          |                   |                  | Buffer Overflow                      |
+|           |                  |          |                   |                  | -->avd.aquasec.com/nvd/cve-2021-3711 |
++-----------+                  +          +                   +                  +                                      +
+| openssl   |                  |          |                   |                  |                                      |
+|           |                  |          |                   |                  |                                      |
+|           |                  |          |                   |                  |                                      |
++-----------+------------------+----------+-------------------+------------------+--------------------------------------+
+
+
+********** END OF scans/image-vulnerabilities-trivy.txt **********
+
+
+----------- SCAN SUMMARY -----------
+Known viruses: 8570581
+Engine version: 0.103.3
+Scanned directories: 1566
+Scanned files: 7410
+Infected files: 0
+Data scanned: 339.71 MB
+Data read: 205.54 MB (ratio 1.65:1)
+Time: 126.512 sec (2 m 6 s)
+Start Date: 2021:10:11 01:13:46
+End Date:   2021:10:11 01:15:52
+
+
+********** END OF scans/recursive-root-dir-clamscan.txt **********
 ```

alpine.clamav.trivy.X9.Dockerfile

@@ -1,21 +1,30 @@
-ARG IMAGE
+ARG REGISTRY
+ARG CLAMAV_IMAGE
+ARG TRIVY_IMAGE
 ARG BASE_IMAGE
+ARG TARGET_IMAGE
 
-FROM $IMAGE as trivy-stage
+FROM $REGISTRY/$CLAMAV_IMAGE as clamav
+FROM $REGISTRY/$TRIVY_IMAGE as trivy
+FROM $REGISTRY/$BASE_IMAGE as base
+
+FROM $REGISTRY/$TARGET_IMAGE as trivy-stage
 ARG TRIVY_SEVERITY
 WORKDIR /scans
+COPY .trivyignore /scans/
+COPY --from=trivy /usr/local/bin/trivy /usr/local/bin/trivy
+RUN trivy filesystem --ignore-unfixed --vuln-type os --severity $TRIVY_SEVERITY --exit-code 0 --no-progress --skip-files usr/local/bin/trivy / | tee image-vulnerabilities-trivy.txt
 
-COPY --from=aquasec/trivy:latest /usr/local/bin/trivy /usr/local/bin/trivy
-RUN trivy filesystem --ignore-unfixed --severity $TRIVY_SEVERITY --exit-code 0 --no-progress --skip-files usr/local/bin/trivy / | tee image-vulnerabilities-trivy.txt
-
-FROM $IMAGE as clamscan-stage
+FROM $REGISTRY/$TARGET_IMAGE as clamscan-stage
 WORKDIR /scans
-
 RUN apk update && apk upgrade && apk add --no-cache clamav-libunrar clamav
+COPY --from=clamav /var/lib/clamav/main.cvd /var/lib/clamav/
+COPY --from=clamav /var/lib/clamav/daily.cvd /var/lib/clamav/
+COPY --from=clamav /var/lib/clamav/bytecode.cvd /var/lib/clamav/
 RUN freshclam
 RUN clamscan -r -i --exclude-dir="^/sys" / >> recursive-root-dir-clamscan.txt
 
-FROM $BASE_IMAGE as final-stage
+FROM base as final-stage
 WORKDIR /scans
 COPY --from=clamscan-stage /scans/recursive-root-dir-clamscan.txt ./recursive-root-dir-clamscan.txt
 COPY --from=trivy-stage /scans/image-vulnerabilities-trivy.txt ./image-vulnerabilities-trivy.txt

debian.clamav.trivy.X9.Dockerfile

@@ -1,21 +1,30 @@
-ARG IMAGE
+ARG REGISTRY
+ARG CLAMAV_IMAGE
+ARG TRIVY_IMAGE
 ARG BASE_IMAGE
+ARG TARGET_IMAGE
 
-FROM $IMAGE as trivy-stage
+FROM $REGISTRY/$CLAMAV_IMAGE as clamav
+FROM $REGISTRY/$TRIVY_IMAGE as trivy
+FROM $REGISTRY/$BASE_IMAGE as base
+
+FROM $REGISTRY/$TARGET_IMAGE as trivy-stage
 ARG TRIVY_SEVERITY
 WORKDIR /scans
+COPY .trivyignore /scans/
+COPY --from=trivy /usr/local/bin/trivy /usr/local/bin/trivy
+RUN trivy filesystem --ignore-unfixed --vuln-type os --severity $TRIVY_SEVERITY --exit-code 0 --no-progress --skip-files usr/local/bin/trivy / | tee image-vulnerabilities-trivy.txt
 
-COPY --from=aquasec/trivy:latest /usr/local/bin/trivy /usr/local/bin/trivy
-RUN trivy filesystem --ignore-unfixed --severity $TRIVY_SEVERITY --exit-code 0 --no-progress --skip-files usr/local/bin/trivy / | tee image-vulnerabilities-trivy.txt
-
-FROM $IMAGE as clamscan-stage
+FROM $REGISTRY/$TARGET_IMAGE as clamscan-stage
 WORKDIR /scans
-
 RUN apt update && apt-get install clamav -y
+COPY --from=clamav /var/lib/clamav/main.cvd /var/lib/clamav/
+COPY --from=clamav /var/lib/clamav/daily.cvd /var/lib/clamav/
+COPY --from=clamav /var/lib/clamav/bytecode.cvd /var/lib/clamav/
 RUN freshclam
 RUN clamscan -r -i --exclude-dir="^/sys" / >> recursive-root-dir-clamscan.txt
 
-FROM $BASE_IMAGE as final-stage
+FROM base as final-stage
 WORKDIR /scans
 COPY --from=clamscan-stage /scans/recursive-root-dir-clamscan.txt ./recursive-root-dir-clamscan.txt
 COPY --from=trivy-stage /scans/image-vulnerabilities-trivy.txt ./image-vulnerabilities-trivy.txt

distroless.clamav.X9.Dockerfile

@@ -1,17 +1,24 @@
-ARG IMAGE
+ARG REGISTRY
+ARG CLAMAV_IMAGE
 ARG BASE_IMAGE
+ARG TARGET_IMAGE
 
-FROM $IMAGE as base
+FROM $REGISTRY/$CLAMAV_IMAGE as clamav
+FROM $REGISTRY/$BASE_IMAGE as base
+FROM $REGISTRY/$TARGET_IMAGE as target
 
-FROM $BASE_IMAGE as base-stage
-COPY --from=base / ../base-root
+FROM base as base-stage
+COPY --from=target / ../base-root
 
 FROM base-stage as clamscan-stage
 WORKDIR /scans
 RUN apk update && apk upgrade && apk add --no-cache clamav-libunrar clamav
+COPY --from=clamav /var/lib/clamav/main.cvd /var/lib/clamav/
+COPY --from=clamav /var/lib/clamav/daily.cvd /var/lib/clamav/
+COPY --from=clamav /var/lib/clamav/bytecode.cvd /var/lib/clamav/
 RUN freshclam
 RUN clamscan -ri /base-root >> recursive-root-dir-clamscan.txt
 
-FROM $BASE_IMAGE as final-stage
+FROM base as final-stage
 WORKDIR /scans
 COPY --from=clamscan-stage /scans/recursive-root-dir-clamscan.txt ./recursive-root-dir-clamscan.txt

distroless.clamav.trivy.X9.Dockerfile

@@ -1,24 +1,34 @@
-ARG IMAGE
+ARG REGISTRY
+ARG CLAMAV_IMAGE
+ARG TRIVY_IMAGE
 ARG BASE_IMAGE
+ARG TARGET_IMAGE
 
-FROM $IMAGE as base
+FROM $REGISTRY/$CLAMAV_IMAGE as clamav
+FROM $REGISTRY/$TRIVY_IMAGE as trivy
+FROM $REGISTRY/$BASE_IMAGE as base
+FROM $REGISTRY/$TARGET_IMAGE as target
 
-FROM $BASE_IMAGE as base-stage
-COPY --from=base / ../base-root
+FROM base as base-stage
+COPY --from=target / ../base-root
 
 FROM base-stage as trivy-stage
 ARG TRIVY_SEVERITY
 WORKDIR /scans
-COPY --from=aquasec/trivy:latest /usr/local/bin/trivy /usr/local/bin/trivy
-RUN trivy filesystem --ignore-unfixed --severity $TRIVY_SEVERITY --exit-code 0 --no-progress /base-root | tee image-vulnerabilities-trivy.txt
+COPY .trivyignore /scans/
+COPY --from=trivy /usr/local/bin/trivy /usr/local/bin/trivy
+RUN trivy filesystem --ignore-unfixed --vuln-type os --severity $TRIVY_SEVERITY --exit-code 0 --no-progress /base-root | tee image-vulnerabilities-trivy.txt
 
 FROM base-stage as clamscan-stage
 WORKDIR /scans
 RUN apk update && apk upgrade && apk add --no-cache clamav-libunrar clamav
+COPY --from=clamav /var/lib/clamav/main.cvd /var/lib/clamav/
+COPY --from=clamav /var/lib/clamav/daily.cvd /var/lib/clamav/
+COPY --from=clamav /var/lib/clamav/bytecode.cvd /var/lib/clamav/
 RUN freshclam
 RUN clamscan -ri /base-root >> recursive-root-dir-clamscan.txt
 
-FROM $BASE_IMAGE as final-stage
+FROM base as final-stage
 WORKDIR /scans
 COPY --from=clamscan-stage /scans/recursive-root-dir-clamscan.txt ./recursive-root-dir-clamscan.txt
 COPY --from=trivy-stage /scans/image-vulnerabilities-trivy.txt ./image-vulnerabilities-trivy.txt