Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ignore] POC dependabot #122

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Auto test new nginx versions

on:
schedule:
- cron: '0 0 * * 0' # Every Sunday at midnight
pull_request_target:
branches: [master, main]
workflow_dispatch:
push:

jobs:
test_nginx_version:
runs-on: ubuntu-22.04
permissions:
actions: read # read secrets
contents: write # Creates a branch
pull-requests: write # Creates a PR
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install Python
uses: actions/setup-python@v4
with:
python--version: '3.x'
- name: Check and Modify
run: echo "NGINX_VERSION_TO_TEST=$(python bin/nginx_dependencies.py)" >> $GITHUB_ENV
- name: Create Pull Request
id: pr
uses: peter-evans/[email protected]
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: "bot/test-nginx-version-$NGINX_VERSION_TO_TEST"
commit-message: "[Test Package Versions Bump]"
delete-branch: true
base: master
title: "[IGNORE] Test new version of NGINX: $NGINX_VERSION_TO_TEST"
reviewers: "DataDog/dd-trace-cpp"
body: |
Test a new release of NGINX

263 changes: 263 additions & 0 deletions bin/base_config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,263 @@
---
version: 2.1
orbs:
codecov: codecov/[email protected]

parameters:
build_all:
type: boolean
default: false

executors:
docker-amd64:
parameters:
image:
type: string
docker:
- image: "<< parameters.image >>"
resource_class: xlarge
docker-arm64:
parameters:
image:
type: string
docker:
- image: "<< parameters.image >>"
resource_class: arm.xlarge

jobs:
build_amd64:
parameters:
nginx-version:
type: string
waf:
type: enum
enum:
- 'ON'
- 'OFF'
steps:
- checkout
- run: git submodule sync && git submodule update --init --recursive
- run:
name: Verify versions (release tag only)
command: |
if [[ $CIRCLE_TAG =~ ^v ]]; then
bin/verify_version.sh "$(echo "$CIRCLE_TAG" | tr -d v)"
else
echo "Not a release"
fi
- run:
command: 'make build-musl'
environment:
BUILD_TYPE: RelWithDebInfo
NGINX_VERSION: "<< parameters.nginx-version >>"
- persist_to_workspace:
root: "."
paths:
- ".musl-build/ngx_http_datadog_module.so"
- ".musl-build/ngx_http_datadog_module.so.debug"
- store_artifacts:
path: ".musl-build/ngx_http_datadog_module.so"
destination: ngx_http_datadog_module.so
- store_artifacts:
path: ".musl-build/ngx_http_datadog_module.so.debug"
destination: ngx_http_datadog_module.so.debug
machine:
image: ubuntu-2204:current
resource_class: xlarge
environment:
ARCH: x86_64
MAKE_JOB_COUNT: 8
WAF: "<< parameters.waf >>"
NGINX_VERSION: "<< parameters.nginx-version >>"
build_arm64:
parameters:
nginx-version:
type: string
waf:
type: enum
enum:
- 'ON'
- 'OFF'
steps:
- checkout
- run: git submodule sync && git submodule update --init --recursive
- run:
command: 'make build-musl'
environment:
BUILD_TYPE: RelWithDebInfo
NGINX_VERSION: "<< parameters.nginx-version >>"
- persist_to_workspace:
root: "."
paths:
- ".musl-build/ngx_http_datadog_module.so"
- ".musl-build/ngx_http_datadog_module.so.debug"
- store_artifacts:
path: ".musl-build/ngx_http_datadog_module.so"
destination: ngx_http_datadog_module.so
- store_artifacts:
path: ".musl-build/ngx_http_datadog_module.so.debug"
destination: ngx_http_datadog_module.so.debug
- store_artifacts:
path: nginx-version-info
destination: nginx-version-info
machine:
image: ubuntu-2204:current
resource_class: arm.xlarge
environment:
ARCH: aarch64
MAKE_JOB_COUNT: 8
WAF: "<< parameters.waf >>"
NGINX_VERSION: "<< parameters.nginx-version >>"
coverage:
environment:
DOCKER_BUILDKIT: 1
steps:
- checkout
- run: git submodule sync && git submodule update --init --recursive
- run: echo -e "ARCH=amd64\nBASE_IMAGE=nginx:{NGINX_VERSION}\n" > nginx-version-info
- run:
command: 'make coverage'
environment:
ARCH: x86_64
MAKE_JOB_COUNT: 8
BUILD_TYPE: RelWithDebInfo
NGINX_VERSION: {NGINX_VERSION}
WAF: ON
- codecov/upload:
upload_args: '--disable-search'
file: .musl-build/coverage.lcov
upload_name: circleci
machine:
image: ubuntu-2204:current
resource_class: xlarge
test:
parameters:
base-image:
type: string
default: ''
nginx-version:
type: string
arch:
type: string
waf:
type: string
executor:
name: docker-<< parameters.arch >>
image: cimg/python:3.10.13
environment:
# https://github.com/containers/podman/issues/13889
DOCKER_BUILDKIT: 1
WAF: "<< parameters.waf >>"
steps:
- checkout
- attach_workspace:
at: "/tmp/workspace"
- run: mv -v /tmp/workspace/.musl-build/ngx_http_datadog_module.so* test/services/nginx/
- run: printf "ARCH=%s\nBASE_IMAGE=%s\n" << parameters.arch >> << parameters.base-image >> > nginx-version-info
- setup_remote_docker:
docker_layer_caching: true
- run: test/bin/run --verbose --failfast
- store_artifacts:
path: test/logs/test.log
destination: test.log
system_tests:
machine:
# https://support.circleci.com/hc/en-us/articles/360007324514-How-can-I-use-Docker-volume-mounting-on-CircleCI-
image: ubuntu-2004:current
resource_class: large
steps:
- attach_workspace:
at: "/tmp/workspace"
- run:
name: clone system-tests repo
command: git clone https://github.com/DataDog/system-tests.git
- run:
name: Install python 3.9
command: sudo apt-get install python3.9-venv
- run:
name: Move the module to the system-tests directory
working_directory: ./system-tests
command: cp /tmp/workspace/.musl-build/ngx_http_datadog_module.so binaries/ngx_http_datadog_module-appsec-amd64-{NGINX_VERSION}.so
- run:
name: Build test targets
working_directory: ./system-tests
command: ./build.sh cpp
- run:
name: Run DEFAULT scenarios
working_directory: ./system-tests
command: ./run.sh
environment:
DD_API_KEY: fakekey
format:
docker:
- image: datadog/docker-library:dd-trace-cpp-ci
resource_class: small
steps:
- checkout
- run:
name: Install Python dependencies
command: |
pip install yapf
update-alternatives --install /usr/local/bin/yapf3 yapf3 /usr/local/bin/yapf 100
- run: make lint
shellcheck:
docker:
- image: koalaman/shellcheck-alpine:v0.9.0
entrypoint: "/bin/sh"
steps:
- checkout
- run: find bin/ test/ example/ installer/ -type f -executable | xargs shellcheck --exclude
SC1071,SC1091,SC2317
workflows:
build-and-test:
when:
and:
- not: << pipeline.git.tag >>
- not: << pipeline.parameters.build_all >>
jobs:
- format
- shellcheck:
name: run shellcheck on shell scripts
- build_amd64:
matrix:
parameters:
nginx-version:
- {NGINX_VERSION}
waf:
- 'ON'
- 'OFF'
name: build << matrix.nginx-version >> on amd64 WAF << matrix.waf >>
- build_arm64:
matrix:
parameters:
nginx-version:
- {NGINX_VERSION}
waf:
- 'ON'
- 'OFF'
name: build << matrix.nginx-version >> on arm64 WAF << matrix.waf >>
- coverage:
name: Coverage on {NGINX_VERSION} with WAF ON
- test:
matrix:
parameters:
arch:
- amd64
- arm64
waf:
- 'ON'
- 'OFF'
base-image:
- nginx:{NGINX_VERSION}-alpine
- nginx:{NGINX_VERSION}
nginx-version:
- {NGINX_VERSION}
name: test << matrix.nginx-version >> on << matrix.base-image >>:<< matrix.arch
>> WAF << matrix.waf >>
requires:
- build << matrix.nginx-version >> on << matrix.arch >> WAF << matrix.waf
>>
- system_tests:
name: Run system tests
requires:
- build {NGINX_VERSION} on amd64 WAF ON
38 changes: 38 additions & 0 deletions bin/nginx_dependencies.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import subprocess
import os

LATEST_VERSION_SUPPORTED = "1.26.0"

def if_version_supported(version):
return tuple(map(int, version.split('.'))) > tuple(map(int, LATEST_VERSION_SUPPORTED.split('.')))

def update_nginx_version(source_file, destination_file, new_version):
with open(source_file, 'r') as file:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Code Quality Violation

Suggested change
with open(source_file, 'r') as file:
with open(source_file) as file:
file with read-only defined (...read more)

If a flag is opened for read-only, no need to put a flag to open the file.

Learn More

View in Datadog  Leave us feedback  Documentation

file_content = file.read()

updated_content = file_content.replace("{NGINX_VERSION}", new_version)

with open(destination_file, 'w') as file:
file.write(updated_content)

# set the pwd in the bin folder
abspath = os.path.abspath(__file__)
dname = os.path.dirname(abspath)
os.chdir(dname)

get_nginx_release_versions = subprocess.Popen(['bash', "nginx_release_downloads.sh"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = get_nginx_release_versions.communicate()

if get_nginx_release_versions.returncode != 0:
print("Could not get nginx versions:", stderr.decode())
exit()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Code Quality Violation

do not use exit() (...read more)

Use sys.exit() instead of exit(). Exit is a builtin and done mostly for the console. sys.exit() is done for program with a proper return argument (see documentation).

Learn More

View in Datadog  Leave us feedback  Documentation



for line in stdout.decode().splitlines():
version, link = line.split(' ', 1)

if if_version_supported(version):
print("export NGINX_VERSION_TO_TEST="+version)
update_nginx_version("base_config.yml", "config.yml", version)
subprocess.run(["mv config.yml ../.circleci/config.yml"], shell=True, check=True)
break
Loading