Establish an e2e backend instance locally and in CI/CD #2429
Conversation
github-actions
bot
added
backend
docker
|
Warning Rate limit exceeded@arkid15r has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 6 minutes and 58 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings. WalkthroughAdds an end-to-end testing setup: new Makefile e2e targets, a docker-compose e2e stack and env examples, a composite GitHub Action to prepare the e2e environment, CI workflow updates to include Postgres and a setup step, and small Dockerfile/compose entrypoint adjustments plus settings and utils changes to support E2E mode. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (7)
docker-compose/backend.e2e.yaml (4)
16-22: Set DJANGO_SETTINGS_MODULE default for e2e.Make the e2e stack independent of dev/prod .env.
environment: + DJANGO_SETTINGS_MODULE: ${DJANGO_SETTINGS_MODULE:-settings.e2e} DJANGO_DB_HOST: ${DJANGO_DB_HOST:-db} DJANGO_DB_NAME: ${DJANGO_DB_NAME:-nest_db_e2e} DJANGO_DB_PASSWORD: ${DJANGO_DB_PASSWORD:-nest_user_e2e_password} DJANGO_DB_PORT: ${DJANGO_DB_PORT:-5432} DJANGO_DB_USER: ${DJANGO_DB_USER:-nest_user_e2e}
22-25: Add a backend healthcheck to enable compose --wait and reliable test gating.Without a healthcheck, CI can race the server startup.
ports: - 8000:8000 + healthcheck: + test: ["CMD-SHELL", "python -c 'import socket,time,sys; \ +for _ in range(30): \ + try: socket.create_connection((\"localhost\",8000),2).close(); sys.exit(0) \ + except OSError: time.sleep(1) \ +sys.exit(1)'"] + interval: 5s + timeout: 3s + retries: 10 + start_period: 10s
41-43: E2E DB volume persists across runs; add a clean target to avoid stale state.Persistent volume improves speed but harms repeatability. Add a Makefile clean target that runs
docker compose -p nest-e2e -f docker-compose/backend.e2e.yaml down -v.
17-21: Default DB creds committed (ok for test) — but prefer .env.e2e to avoid spreading defaults.Low risk, but moving these to an
.env.e2ekeeps compose cleaner and avoids accidental reuse.Also applies to: 29-33
backend/docker/Dockerfile.e2e (2)
23-24: Pin Poetry to a version for reproducible builds.Unpinned Poetry can break builds unexpectedly.
-RUN --mount=type=cache,target=${PIP_CACHE_DIR} \ - python -m pip install poetry --cache-dir ${PIP_CACHE_DIR} +RUN --mount=type=cache,target=${PIP_CACHE_DIR} \ + python -m pip install 'poetry==1.8.3' --cache-dir ${PIP_CACHE_DIR}
31-32: Install only runtime deps for a slimmer image.If dev deps exist, prefer:
poetry install --no-root --only main.- poetry install --no-root + poetry install --no-root --only mainbackend/Makefile (1)
14-20: Add a clean target for the e2e stack (containers + volume).Ensures repeatable runs and easy teardown.
clean-backend-docker: @docker container rm -f nest-backend >/dev/null 2>&1 || true @docker container rm -f nest-cache >/dev/null 2>&1 || true @docker container rm -f nest-db >/dev/null 2>&1 || true @docker image rm -f nest-local-backend >/dev/null 2>&1 || true @docker volume rm -f nest-local_backend-venv >/dev/null 2>&1 || true + +clean-backend-e2e-docker: + @docker compose --project-name nest-e2e -f docker-compose/backend.e2e.yaml down -v
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
backend/data/nest-e2e-data.sql.gzis excluded by!**/*.gz
📒 Files selected for processing (3)
backend/Makefile(4 hunks)backend/docker/Dockerfile.e2e(1 hunks)docker-compose/backend.e2e.yaml(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run backend tests
- GitHub Check: Run frontend unit tests
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (2)
backend/Makefile (1)
84-87: load-data-e2e is fine as long as backend is healthy.No changes needed; verify it follows
run-backend-e2ein CI to avoid races.backend/docker/Dockerfile.e2e (1)
1-1: Ensure registry mirror sync and BuildKit enabled
python:3.13.7-alpineexists on Docker Hub—verify your mirror has this tag- CI environment didn’t expose BuildKit variables—enable BuildKit (
DOCKER_BUILDKIT=1) forRUN --mountsupport
github-actions
bot
added
ci
and removed
docker
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
.github/workflows/run-ci-cd.yaml(1 hunks)backend/docker/Dockerfile(0 hunks)docker-compose/backend.e2e.yaml(1 hunks)docker-compose/production.yaml(1 hunks)docker-compose/staging.yaml(1 hunks)
💤 Files with no reviewable changes (1)
- backend/docker/Dockerfile
🚧 Files skipped from review as they are similar to previous changes (1)
- docker-compose/backend.e2e.yaml
🧰 Additional context used
🪛 actionlint (1.7.8)
.github/workflows/run-ci-cd.yaml
188-188: unexpected key "environment" for "services" section. expected one of "credentials", "env", "image", "options", "ports", "volumes"
(syntax-check)
🔇 Additional comments (3)
docker-compose/production.yaml (1)
4-4: LGTM — verify entrypoint script exists in the backend image.The explicit entrypoint aligns with the backend Dockerfile changes and provides consistent startup across environments.
Confirm that
/home/owasp/entrypoint.shis present in the backend Docker image (backend/docker/Dockerfile context).docker-compose/staging.yaml (1)
4-4: LGTM — mirrors production entrypoint.Consistent with production.yaml and backend Dockerfile updates.
.github/workflows/run-ci-cd.yaml (1)
233-238: The e2e test data file is properly committed and available.The file
backend/data/nest-e2e-data.sql.gzis tracked in git (commit 4002a19) and present on the PR branch. The workflow will successfully access it during execution. No action needed.
arkid15r
left a comment
arkid15r
left a comment
There was a problem hiding this comment.
It looks like the approach is nearly right -- we should create a PG instance for jobs we want to use it in, e.g. fuzzing and e2e testing jobs. Let's discuss it in Slack.
github-actions
bot
added
docs
arkid15r
left a comment
arkid15r
left a comment
There was a problem hiding this comment.
Great start, let's shape it further 🚀
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
.github/workflows/setup-e2e-environment/action.yaml (2)
9-9: Critical: Hardcoded database credentials must be parameterized.Database credentials (username, database name) are hardcoded in the
pg_isreadycommand. These should be accepted as action inputs to allow flexibility and avoid embedding credentials in source control.Based on prior unresolved feedback, add action inputs:
+inputs: + db-user: + description: 'PostgreSQL user for e2e environment' + required: false + default: 'nest_user_e2e' + db-name: + description: 'PostgreSQL database name for e2e environment' + required: false + default: 'nest_db_e2e' + db-password: + description: 'PostgreSQL password (use GitHub secret)' + required: true + runs: using: 'composite' steps: - name: Wait for database to be ready run: | - until docker exec ${{ job.services.db.id }} pg_isready -U nest_user_e2e -d nest_db_e2e; do + until docker exec ${{ job.services.db.id }} pg_isready -U ${{ inputs.db-user }} -d ${{ inputs.db-name }}; do
19-24: Critical: Remove hardcoded database password.Line 21 embeds the database password
nest_user_e2e_passwordin source control. This is a security vulnerability—passwords must never be committed to a repository, even for testing.Use the
db-passwordinput from the previous suggestion:- name: Load Postgres data env: - PGPASSWORD: nest_user_e2e_password + PGPASSWORD: ${{ inputs.db-password }} run: | - gunzip -c backend/data/nest-e2e.sql.gz | psql -h localhost -U nest_user_e2e -d nest_db_e2e + gunzip -c backend/data/nest-e2e.sql.gz | psql -h localhost -U ${{ inputs.db-user }} -d ${{ inputs.db-name }}Then update the calling workflow to pass the secret:
- name: Setup E2E environment uses: ./.github/workflows/setup-e2e-environment with: db-user: nest_user_e2e db-name: nest_db_e2e db-password: ${{ secrets.NEST_E2E_DB_PASSWORD }}Ensure
NEST_E2E_DB_PASSWORDis added to GitHub repository secrets.backend/Makefile (1)
64-66: Critical: Shell redirection in CMD variable will not execute correctly.The pipeline with
|and>in the CMD variable will not be interpreted properly by Make. The redirection occurs inside the container's context, and the shell metacharacters may not expand as expected. Theexec-db-e2e-commandtarget passes CMD todocker exec, but the redirection should happen on the host side.Run the dump pipeline from the host:
dump-data-e2e: @echo "Dumping Nest e2e data" - @CMD="pg_dumpall -U nest_user_e2e --clean | gzip -9 > backend/data/nest-e2e.sql.gz" $(MAKE) exec-db-e2e-command + @mkdir -p backend/data + @docker exec -i nest-db-e2e pg_dumpall -U nest_user_e2e --clean | gzip -9 > backend/data/nest-e2e.sql.gzNote: Removed
-tto avoid TTY artifacts in the dump, and removed-hto use the local socket.
🧹 Nitpick comments (2)
.github/workflows/setup-e2e-environment/action.yaml (2)
39-50: Backend startup mixes env file with explicit overrides.Line 42 loads
backend/.env.e2e.examplebut line 44 explicitly overridesDJANGO_DB_HOST=localhost. If the env file already contains the correct values, the explicit override is redundant. If different values are needed, consider accepting them as action inputs for flexibility.Consider accepting database connection parameters as inputs and removing the hardcoded override, or document why
DJANGO_DB_HOSTmust be overridden despite using an env file.
52-59: Use curl instead of wget for consistency.Based on previous feedback noting "wget/curl inconsistent usage", consider standardizing on one tool. The codebase may prefer
curlfor HTTP checks.- name: Waiting for the backend to be ready run: | - until wget --spider http://localhost:8000/a; do + until curl -f -s http://localhost:8000/a > /dev/null; do echo "Waiting for backend..." sleep 3 done
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
.github/workflows/setup-e2e-environment/action.yaml(1 hunks).gitignore(1 hunks)backend/.env.e2e.example(1 hunks)backend/Makefile(4 hunks)backend/apps/common/utils.py(1 hunks)backend/settings/base.py(1 hunks)backend/settings/e2e.py(1 hunks)docker-compose/e2e.yaml(1 hunks)frontend/.env.e2e.example(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- docker-compose/e2e.yaml
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 2429
File: backend/Makefile:30-32
Timestamp: 2025-10-26T12:50:50.512Z
Learning: The `exec-backend-e2e-command` and `exec-db-e2e-command` Makefile targets in the backend/Makefile are intended for local development and debugging only, not for CI/CD execution, so the `-it` flags are appropriate.
📚 Learning: 2025-10-26T12:50:50.512Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 2429
File: backend/Makefile:30-32
Timestamp: 2025-10-26T12:50:50.512Z
Learning: The `exec-backend-e2e-command` and `exec-db-e2e-command` Makefile targets in the backend/Makefile are intended for local development and debugging only, not for CI/CD execution, so the `-it` flags are appropriate.
Applied to files:
backend/Makefile
🧬 Code graph analysis (1)
backend/settings/e2e.py (1)
backend/settings/base.py (1)
Base(9-213)
🪛 actionlint (1.7.8)
.github/workflows/setup-e2e-environment/action.yaml
1-1: "on" section is missing in workflow
(syntax-check)
1-1: "jobs" section is missing in workflow
(syntax-check)
2-2: unexpected key "description" for "workflow" section. expected one of "concurrency", "defaults", "env", "jobs", "name", "on", "permissions", "run-name"
(syntax-check)
4-4: unexpected key "runs" for "workflow" section. expected one of "concurrency", "defaults", "env", "jobs", "name", "on", "permissions", "run-name"
(syntax-check)
🪛 dotenv-linter (4.0.0)
frontend/.env.e2e.example
[warning] 15-15: [UnorderedKey] The NEXTAUTH_SECRET key should go before the NEXT_PUBLIC_API_URL key
(UnorderedKey)
[warning] 16-16: [UnorderedKey] The NEXTAUTH_URL key should go before the NEXT_PUBLIC_API_URL key
(UnorderedKey)
backend/.env.e2e.example
[warning] 8-8: [UnorderedKey] The DJANGO_CONFIGURATION key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 9-9: [UnorderedKey] The DJANGO_DB_HOST key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 10-10: [UnorderedKey] The DJANGO_DB_NAME key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 11-11: [UnorderedKey] The DJANGO_DB_USER key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 12-12: [UnorderedKey] The DJANGO_DB_PASSWORD key should go before the DJANGO_DB_USER key
(UnorderedKey)
[warning] 13-13: [UnorderedKey] The DJANGO_DB_PORT key should go before the DJANGO_DB_USER key
(UnorderedKey)
[warning] 14-14: [UnorderedKey] The DJANGO_OPEN_AI_SECRET_KEY key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 15-15: [QuoteCharacter] The value has quote characters (', ")
(QuoteCharacter)
[warning] 15-15: [UnorderedKey] The DJANGO_PUBLIC_IP_ADDRESS key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 16-16: [UnorderedKey] The DJANGO_REDIS_HOST key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 17-17: [UnorderedKey] The DJANGO_REDIS_PASSWORD key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 18-18: [UnorderedKey] The DJANGO_RELEASE_VERSION key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 19-19: [UnorderedKey] The DJANGO_SECRET_KEY key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 20-20: [UnorderedKey] The DJANGO_SENTRY_DSN key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
🔇 Additional comments (6)
.gitignore (1)
9-9: LGTM!The change correctly unignores
.env.e2e.examplefiles, following the same pattern as the existing.env.exampleunignore rule on line 8.frontend/.env.e2e.example (1)
1-16: LGTM!The e2e example environment file is well-structured with appropriate placeholder values for local testing. The localhost URLs and placeholder secrets are suitable for an example file that developers will copy and customize.
backend/settings/base.py (1)
22-22: LGTM!The new
IS_E2E_ENVIRONMENTflag follows the established pattern of other environment indicators (lines 23-26) and provides a clean way to detect e2e testing contexts throughout the application.backend/Makefile (1)
116-118: Running in foreground may block in CI contexts.The
docker compose upcommand runs in the foreground with--abort-on-container-exit. While suitable for local development (where blocking is expected), this would block CI pipelines if this target is ever used in automation.Verify whether this target will be invoked by CI/CD pipelines. If yes, consider adding a detached variant:
run-backend-e2e-detached: @DOCKER_BUILDKIT=1 \ docker compose --project-name nest-e2e -f docker-compose/e2e.yaml up -d --build --wait --remove-orphans backend dbIf this target is strictly for local development, document that in a comment.
backend/apps/common/utils.py (1)
98-98: LGTM!The updated logic correctly treats e2e environments like local environments for IP address resolution. In both cases, returning
PUBLIC_IP_ADDRESSis appropriate since the request context won't have production-like forwarding headers.backend/settings/e2e.py (1)
1-23: LGTM with observation: Consider whether disabling all logging is intentional.The E2E configuration is well-structured and correctly sets
IS_E2E_ENVIRONMENT = True(line 21) andDEBUG = False(line 20) for production-like testing. The CORS and CSRF origins appropriately cover both Docker service names and localhost URLs.However, line 22 sets
LOGGING = {}, which completely disables the logging configuration inherited fromBase(Base.LOGGING includes console handler and django logger). This may make debugging e2e test failures more difficult.Verify whether completely disabling logging is intentional for e2e tests. If test output needs to remain clean, consider setting log levels to WARNING/ERROR rather than removing all logging configuration:
LOGGING = { "version": 1, "disable_existing_loggers": False, "handlers": { "console": { "level": "WARNING", "class": "logging.StreamHandler", }, }, "loggers": { "django": { "handlers": ["console"], "level": "WARNING", }, }, }
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
frontend/.env.e2e.example (1)
1-16: Consider adding inline comments or a header explaining the e2e environment setup.This example file would benefit from a brief comment block explaining its purpose and which variables are critical for the e2e tests to function. This aligns with the PR objective to "provide documentation explaining how to use the new test environment."
Example header:
+# E2E Testing Environment Configuration +# This file defines environment variables for end-to-end and integration tests. +# Copy this file to .env.e2e and update placeholders (GitHub OAuth, NEXTAUTH_SECRET). +# Backend must be running on port 8001 for tests to work. + NEXT_PUBLIC_API_URL=http://localhost:8001/
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
CONTRIBUTING.md(1 hunks)backend/Makefile(4 hunks)backend/settings/e2e.py(1 hunks)docker-compose/e2e.yaml(1 hunks)frontend/.env.e2e.example(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
- backend/Makefile
- CONTRIBUTING.md
- docker-compose/e2e.yaml
- backend/settings/e2e.py
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 2429
File: backend/Makefile:30-32
Timestamp: 2025-10-26T12:50:50.512Z
Learning: The `exec-backend-e2e-command` and `exec-db-e2e-command` Makefile targets in the backend/Makefile are intended for local development and debugging only, not for CI/CD execution, so the `-it` flags are appropriate.
🪛 dotenv-linter (4.0.0)
frontend/.env.e2e.example
[warning] 15-15: [UnorderedKey] The NEXTAUTH_SECRET key should go before the NEXT_PUBLIC_API_URL key
(UnorderedKey)
[warning] 16-16: [UnorderedKey] The NEXTAUTH_URL key should go before the NEXT_PUBLIC_API_URL key
(UnorderedKey)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run frontend unit tests
- GitHub Check: Run backend tests
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (2)
frontend/.env.e2e.example (2)
3-3: Verify the environment label for e2e tests.
NEXT_PUBLIC_ENVIRONMENT=localmay be semantically misleading for e2e testing context. Consider whether this should be set to"e2e"to clearly distinguish end-to-end test runs from local development, which could be important for conditional logic or analytics/monitoring.
1-4: Port and environment configuration verified as correct.All backend endpoints in
frontend/.env.e2e.examplecorrectly targethttp://localhost:8001, which aligns with the docker-compose e2e setup (docker-compose/e2e.yamlport mapping8001:8000). TheNEXT_PUBLIC_ENVIRONMENT=localsetting is intentionally configured for e2e tests, as confirmed by the Makefile and its usage innext.config.ts.
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (2)
.github/workflows/setup-e2e-environment/action.yaml (2)
8-13: Parameterize hardcoded database credentials.The database username and database name remain hardcoded in the readiness check. This issue was flagged in previous reviews but remains unresolved. Add action
inputsfor these values:+inputs: + db-user: + description: 'PostgreSQL user for E2E environment' + required: false + default: 'nest_user_e2e' + db-name: + description: 'PostgreSQL database name for E2E environment' + required: false + default: 'nest_db_e2e' + db-password: + description: 'PostgreSQL password (use GitHub secret)' + required: true + runs: using: 'composite' steps: - name: Wait for database to be ready run: | - until docker exec ${{ job.services.db.id }} pg_isready -U nest_user_e2e -d nest_db_e2e; do + until docker exec ${{ job.services.db.id }} pg_isready -U ${{ inputs.db-user }} -d ${{ inputs.db-name }}; do
18-22: Critical: Remove hardcoded database password from source control.Line 20 contains a hardcoded password (
nest_user_e2e_password) committed to the repository. This is a security vulnerability regardless of whether it's for testing purposes. Credentials must never be stored in source code.This critical issue was flagged in multiple previous review comments but remains unresolved.
+inputs: + db-password: + description: 'PostgreSQL password (pass via GitHub secret)' + required: true - name: Load Postgres data env: - PGPASSWORD: nest_user_e2e_password + PGPASSWORD: ${{ inputs.db-password }} run: | - gunzip -c backend/data/nest-e2e.sql.gz | psql -h localhost -U nest_user_e2e -d nest_db_e2e + gunzip -c backend/data/nest-e2e.sql.gz | psql -h localhost -U ${{ inputs.db-user }} -d ${{ inputs.db-name }}Update the caller workflow to pass the secret:
- name: Setup E2E environment uses: ./.github/workflows/setup-e2e-environment with: db-user: nest_user_e2e db-name: nest_db_e2e db-password: ${{ secrets.NEST_E2E_DB_PASSWORD }}Ensure
NEST_E2E_DB_PASSWORDis added to GitHub repository secrets.
🧹 Nitpick comments (2)
frontend/.env.e2e.example (1)
3-3: Consider using "e2e" for the environment identifier.The
NEXT_PUBLIC_ENVIRONMENTis set to "local", which might be confusing in an E2E-specific configuration file. If the E2E environment is distinct from local development, consider changing this to:-NEXT_PUBLIC_ENVIRONMENT=local +NEXT_PUBLIC_ENVIRONMENT=e2eThis would provide clearer distinction between local development and E2E testing environments in logs, analytics, and feature flags.
.github/workflows/setup-e2e-environment/action.yaml (1)
48-53: Consider using curl for consistency.As noted in a previous review comment, there's inconsistent usage of
wgetandcurlacross workflows. For better consistency and simpler syntax, consider usingcurl:- name: Waiting for the backend to be ready run: | - until wget --spider http://localhost:9000/a/; do + until curl -f -s http://localhost:9000/a/ > /dev/null; do echo "Waiting for backend..." sleep 3 done echo "Backend is up!"The
-fflag makes curl return an error on HTTP errors, and-ssuppresses progress output.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
.github/workflows/run-ci-cd.yaml(1 hunks).github/workflows/setup-e2e-environment/action.yaml(1 hunks).gitignore(1 hunks)CONTRIBUTING.md(1 hunks)backend/Makefile(4 hunks)backend/apps/common/utils.py(1 hunks)backend/docker/Dockerfile(0 hunks)backend/settings/base.py(1 hunks)docker-compose/e2e.yaml(1 hunks)docker-compose/production.yaml(1 hunks)docker-compose/staging.yaml(1 hunks)frontend/.env.e2e.example(1 hunks)
💤 Files with no reviewable changes (1)
- backend/docker/Dockerfile
🚧 Files skipped from review as they are similar to previous changes (7)
- .gitignore
- backend/apps/common/utils.py
- docker-compose/e2e.yaml
- CONTRIBUTING.md
- .github/workflows/run-ci-cd.yaml
- docker-compose/staging.yaml
- backend/Makefile
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 2429
File: backend/Makefile:30-32
Timestamp: 2025-10-26T12:50:50.512Z
Learning: The `exec-backend-e2e-command` and `exec-db-e2e-command` Makefile targets in the backend/Makefile are intended for local development and debugging only, not for CI/CD execution, so the `-it` flags are appropriate.
📚 Learning: 2025-10-17T15:25:55.624Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2431
File: infrastructure/providers.tf:1-3
Timestamp: 2025-10-17T15:25:55.624Z
Learning: The infrastructure code in the OWASP/Nest repository (infrastructure/ directory) is intended for quick testing purposes only, not for production deployment.
Applied to files:
docker-compose/production.yaml
🪛 actionlint (1.7.8)
.github/workflows/setup-e2e-environment/action.yaml
1-1: "on" section is missing in workflow
(syntax-check)
1-1: "jobs" section is missing in workflow
(syntax-check)
3-3: unexpected key "description" for "workflow" section. expected one of "concurrency", "defaults", "env", "jobs", "name", "on", "permissions", "run-name"
(syntax-check)
5-5: unexpected key "runs" for "workflow" section. expected one of "concurrency", "defaults", "env", "jobs", "name", "on", "permissions", "run-name"
(syntax-check)
🪛 dotenv-linter (4.0.0)
frontend/.env.e2e.example
[warning] 15-15: [UnorderedKey] The NEXTAUTH_SECRET key should go before the NEXT_PUBLIC_API_URL key
(UnorderedKey)
[warning] 16-16: [UnorderedKey] The NEXTAUTH_URL key should go before the NEXT_PUBLIC_API_URL key
(UnorderedKey)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run frontend unit tests
- GitHub Check: Run backend tests
- GitHub Check: CodeQL (python)
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (2)
backend/settings/base.py (1)
22-22: LGTM!The
IS_E2E_ENVIRONMENTflag follows the established pattern for environment type indicators and integrates cleanly with the existing configuration structure..github/workflows/setup-e2e-environment/action.yaml (1)
24-35: LGTM!The Docker build configuration properly uses GitHub Actions cache, targets the correct platform, and loads the image for subsequent steps.
| services: | ||
| production-nest-backend: | ||
| container_name: production-nest-backend | ||
| entrypoint: /home/owasp/entrypoint.sh |
There was a problem hiding this comment.
🧩 Analysis chain
Verify the entrypoint script exists and is properly configured.
The production service now explicitly calls /home/owasp/entrypoint.sh. Ensure this script:
- Exists in the Docker image at the specified path
- Is executable (
chmod +x) - Properly handles signals for graceful shutdown (especially SIGTERM)
- Works correctly with the depends_on health checks for cache and database services
Run the following script to verify the entrypoint is present in the backend image:
🏁 Script executed:
#!/bin/bash
# Verify entrypoint script exists in the backend production image
docker run --rm --entrypoint ls owasp/nest:backend-production -la /home/owasp/entrypoint.sh 2>&1 || echo "Entrypoint script not found or image not available locally"Length of output: 103
🏁 Script executed:
#!/bin/bash
# Search for entrypoint.sh in the repository
fd -t f entrypoint.shLength of output: 28
🏁 Script executed:
#!/bin/bash
# Find Dockerfile for the backend service
fd -t f -e Dockerfile | grep -i "backend\|nest" || fd -t f -e DockerfileLength of output: 0
🏁 Script executed:
#!/bin/bash
# Search for references to the entrypoint path in docker-compose and Dockerfiles
rg -n "entrypoint.sh|/home/owasp" --type yaml --type dockerfileLength of output: 34
🏁 Script executed:
#!/bin/bash
# Examine the entrypoint script content
cat backend/docker/entrypoint.shLength of output: 160
🏁 Script executed:
#!/bin/bash
# Find all Dockerfiles (remove type filter to avoid errors)
fd -t f "Dockerfile" -o "Dockerfile.*"Length of output: 143
🏁 Script executed:
#!/bin/bash
# Look for Docker-related files in backend/docker directory
ls -la backend/docker/ 2>/dev/null || echo "Directory not found"Length of output: 365
🏁 Script executed:
#!/bin/bash
# Search for COPY and ADD commands that reference entrypoint in any files
rg -n "COPY|ADD" -A 1 -B 1 | grep -A 2 -B 2 "entrypoint\|/home/owasp"Length of output: 1355
🏁 Script executed:
#!/bin/bash
# View full backend/docker/Dockerfile to understand working directory and path resolution
cat backend/docker/DockerfileLength of output: 1731
🏁 Script executed:
#!/bin/bash
# Check if entrypoint.sh is marked executable in git
git ls-files -s backend/docker/entrypoint.shLength of output: 78
Add signal handlers to entrypoint script for graceful shutdown in production.
The script exists at /home/owasp/entrypoint.sh and is executable. However, it lacks signal handling for SIGTERM, which is critical for graceful shutdown. The script runs gunicorn directly without exec or trap handlers, meaning container shutdown signals won't propagate to the application.
Location: backend/docker/entrypoint.sh
Fix: Add signal forwarding before gunicorn:
#!/bin/sh
set -e
python manage.py migrate
python manage.py collectstatic --noinput
python manage.py clear_cache
exec gunicorn wsgi:application --bind 0.0.0.0:8000The exec prefix ensures gunicorn receives signals directly (SIGTERM/SIGINT) from the container orchestrator for clean termination during deployments and restarts.
🤖 Prompt for AI Agents
In docker-compose/production.yaml around line 4 (entrypoint:
/home/owasp/entrypoint.sh) the referenced entrypoint script
backend/docker/entrypoint.sh lacks signal handling and runs gunicorn without
exec so container signals (SIGTERM/SIGINT) won’t reach the process; update the
entrypoint script to run initialization commands as-is but start gunicorn with
exec so it replaces the shell (exec gunicorn ...) and/or add a lightweight trap
for SIGTERM/SIGINT that forwards the signal to the gunicorn PID before exiting
to ensure graceful shutdown and allow the orchestrator to terminate the app
cleanly.
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
.github/workflows/setup-e2e-environment/action.yaml (1)
8-14: Major security risk: Hardcoded database credentials in wait loop.Lines 10 hardcodes the database username (
nest_user_e2e) and name (nest_db_e2e). Parameterize these via action inputs for flexibility and to align with secrets management best practices.Add action inputs for database configuration:
+inputs: + db-user: + description: 'PostgreSQL user for e2e environment' + required: false + default: 'nest_user_e2e' + db-name: + description: 'PostgreSQL database name for e2e environment' + required: false + default: 'nest_db_e2e' + runs: using: composite steps: - name: Wait for database to be ready run: | - until docker exec ${{ job.services.db.id }} pg_isready -U nest_user_e2e -d nest_db_e2e; do + until docker exec ${{ job.services.db.id }} pg_isready -U ${{ inputs.db-user }} -d ${{ inputs.db-name }}; do echo "Waiting for database..." sleep 5 done shell: bashAlso update the psql command on line 24 to use these inputs:
run: | - gunzip -c backend/data/nest-e2e.sql.gz | psql -h localhost -U nest_user_e2e -d nest_db_e2e + gunzip -c backend/data/nest-e2e.sql.gz | psql -h localhost -U ${{ inputs.db-user }} -d ${{ inputs.db-name }}
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
.github/workflows/setup-e2e-environment/action.yaml(1 hunks)frontend/playwright.config.ts(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 2429
File: backend/Makefile:30-32
Timestamp: 2025-10-26T12:50:50.512Z
Learning: The `exec-backend-e2e-command` and `exec-db-e2e-command` Makefile targets in the backend/Makefile are intended for local development and debugging only, not for CI/CD execution, so the `-it` flags are appropriate.
📚 Learning: 2025-10-26T12:50:50.512Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 2429
File: backend/Makefile:30-32
Timestamp: 2025-10-26T12:50:50.512Z
Learning: The `exec-backend-e2e-command` and `exec-db-e2e-command` Makefile targets in the backend/Makefile are intended for local development and debugging only, not for CI/CD execution, so the `-it` flags are appropriate.
Applied to files:
.github/workflows/setup-e2e-environment/action.yaml
🪛 actionlint (1.7.8)
.github/workflows/setup-e2e-environment/action.yaml
1-1: "on" section is missing in workflow
(syntax-check)
1-1: "jobs" section is missing in workflow
(syntax-check)
3-3: unexpected key "description" for "workflow" section. expected one of "concurrency", "defaults", "env", "jobs", "name", "on", "permissions", "run-name"
(syntax-check)
5-5: unexpected key "runs" for "workflow" section. expected one of "concurrency", "defaults", "env", "jobs", "name", "on", "permissions", "run-name"
(syntax-check)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run backend tests
- GitHub Check: Run frontend unit tests
🔇 Additional comments (2)
.github/workflows/setup-e2e-environment/action.yaml (1)
40-50: Verify environment file adequacy for backend startup.Line 43 loads environment variables from
backend/.env.e2e.example, but there is no validation that all required Django settings are present. If the environment file is incomplete or contains invalid values (e.g.,DJANGO_DB_HOST=Noneas noted in prior feedback), the backend container will fail to start or connect to the database.Please confirm that
backend/.env.e2e.exampleis properly configured with valid database connection parameters and all required Django settings.frontend/playwright.config.ts (1)
32-36: Confirm that a 5‑minute webServer timeout is actually requiredBumping
webServer.timeoutto 300_000 ms is valid and will help with slow e2e startup, but it also means CI can hang for up to 5 minutes before failing if the app never comes up. Please confirm that your measured startup times under load/CI justify this much headroom; if not, consider a lower value so failures surface faster.
| - name: Load Postgres data | ||
| env: | ||
| PGPASSWORD: nest_user_e2e_password | ||
| run: | | ||
| gunzip -c backend/data/nest-e2e.sql.gz | psql -h localhost -U nest_user_e2e -d nest_db_e2e | ||
| shell: bash |
There was a problem hiding this comment.
Critical security risk: Hardcoded database password in source code.
Line 22 embeds the database password nest_user_e2e_password in plaintext. Passwords must never be committed to the repository, even for test environments. Accept the password as a GitHub secret via action input instead.
Define the action input and replace the hardcoded value:
+inputs:
+ db-password:
+ description: 'PostgreSQL password for e2e environment (use GitHub secret)'
+ required: true
+
runs:
using: composite
steps:
- name: Load Postgres data
env:
- PGPASSWORD: nest_user_e2e_password
+ PGPASSWORD: ${{ inputs.db-password }}
run: |
gunzip -c backend/data/nest-e2e.sql.gz | psql -h localhost -U nest_user_e2e -d nest_db_e2eThen update the caller in .github/workflows/run-ci-cd.yaml to pass the secret:
- name: Setup E2E environment
uses: ./.github/workflows/setup-e2e-environment
with:
db-password: ${{ secrets.NEST_E2E_DB_PASSWORD }}Ensure NEST_E2E_DB_PASSWORD is configured in GitHub repository secrets.
🤖 Prompt for AI Agents
In .github/workflows/setup-e2e-environment/action.yaml around lines 20-25 the
Postgres password is hardcoded (security risk); change the action to accept a
db-password input and replace the hardcoded env value with the input reference,
then update the caller in .github/workflows/run-ci-cd.yaml to pass the secret
(e.g. db-password: ${{ secrets.NEST_E2E_DB_PASSWORD }}); finally ensure
NEST_E2E_DB_PASSWORD is set in the repository secrets.
|
arkid15r
left a comment
arkid15r
left a comment
There was a problem hiding this comment.
I'm not sure whether this setup works, as I got Server Error 500 for the :9000 port running locally
docker-compose/e2e.yaml
Outdated
| networks: | ||
| - nest-network | ||
| ports: | ||
| - 8001:8000 |
There was a problem hiding this comment.
8001 is already taken by docs
There was a problem hiding this comment.
How was this dump created and what data is already there?
And more important question -- how are we going to maintain this data and keep it up to date?
| 3. Load the data into the e2e db with the following command (in another terminal session): | ||
|
|
||
| ```bash | ||
| make load-data-e2e |
There was a problem hiding this comment.
This approach works for the initial set up. What about the following e2e dump data updates? Is it compact enough to include the data loading as part of e2e test run process?
| -f frontend/docker/Dockerfile.e2e.test frontend \ | ||
| -t nest-test-frontend-e2e | ||
| @docker run --env-file frontend/.env.example --rm nest-test-frontend-e2e pnpm run test:e2e | ||
| @DOCKER_BUILDKIT=1 NEXT_PUBLIC_ENVIRONMENT=local \ |
There was a problem hiding this comment.
It might be a good time to think about introducing e2e env for the frontend
|
|
||
| - name: Waiting for the backend to be ready | ||
| run: | | ||
| until wget --spider http://localhost:9000/a; do |
There was a problem hiding this comment.
It seems this waits forever (until GH jobs times out). Let's add some reasonable timeout?
2 participants
Proposed change
Resolves #2422
Add the PR description here.
Checklist
make check-testlocally; all checks and tests passed.