Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
3d3e49f
ops: 관측 스택 검증 스크립트와 회귀 테스트를 추가
kyuoogle Mar 25, 2026
d3f5341
ops: 풀스택 콜드스타트 스모크와 API 문서 검증 자동화를 추가
kyuoogle Mar 25, 2026
105332e
ops: 다중 인증 세션 격리 리허설 스크립트를 추가
kyuoogle Mar 25, 2026
0ec4dfa
ops: 롤백 리허설과 go-no-go 체크리스트 자산을 추가
kyuoogle Mar 25, 2026
bdb54a8
test: 10.4 증적 집계 스크립트와 회귀 테스트를 추가
kyuoogle Mar 25, 2026
8d9bef5
ci: 10.4 풀스택 스모크 리허설 워크플로를 추가
kyuoogle Mar 25, 2026
8971c9c
fix: 10.4 리허설 워크플로와 증적 집계를 정합화
kyuoogle Mar 25, 2026
588ffe7
fix: 10.4 풀스택 스모크 준비 대기와 BE 포인터를 정합화
kyuoogle Mar 25, 2026
af68b54
fix: 10.4 증적 계약과 리허설 검증을 정합화
kyuoogle Mar 25, 2026
b95e0ce
fix: 10.4 스모크 루프백 주소와 BE 포인터를 정합화
kyuoogle Mar 25, 2026
6f5ce57
fix: 10.4 BE 서브모듈 포인터 해시를 정합화
kyuoogle Mar 25, 2026
46e259d
fix: 10.4 스모크 mandatory API 경로를 분리
kyuoogle Mar 25, 2026
5b5fbeb
fix: 10.4 스모크 실패 증적을 유지하도록 조정
kyuoogle Mar 25, 2026
5f88ea9
fix: 10.4 스모크 step outcome을 정합화
kyuoogle Mar 25, 2026
2b6371a
fix: mysql grant repair 스크립트 누락을 복구
kyuoogle Mar 25, 2026
b0bab8f
fix: edge health 라우팅을 management 포트로 분리
kyuoogle Mar 25, 2026
86534c9
fix: 스모크 문서 검증과 세션 격리 대기를 조정
kyuoogle Mar 25, 2026
e3c0e39
fix: 관측 검증과 채널 재기동 대기를 보강
kyuoogle Mar 25, 2026
f533330
fix: 세션 격리 리허설의 deadlock 재시도를 추가
kyuoogle Mar 25, 2026
b59aef0
fix: 세션 격리 실행을 직렬화하고 deadlock 재시도를 확대
kyuoogle Mar 25, 2026
e27886e
fix: 세션 격리 주문 세션 step-up 검증을 자동화
kyuoogle Mar 25, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
scripts/observability/validate-observability-stack.sh text eol=lf
scripts/infra-bootstrap/repair-service-databases.sh text eol=lf
scripts/release-readiness/run-edge-gateway-validation.sh text eol=lf
scripts/release-readiness/run-full-stack-smoke.sh text eol=lf
scripts/release-readiness/run-rollback-rehearsal.sh text eol=lf
186 changes: 186 additions & 0 deletions .github/workflows/story-10-4-full-stack-smoke-rehearsal.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
name: Story 10.4 Full-Stack Smoke Rehearsal

on:
pull_request:
branches:
- main
paths:
- 'BE'
- 'BE/**'
- 'docker-compose.yml'
- 'docker/**'
- 'scripts/release-readiness/**'
- 'scripts/observability/**'
- 'scripts/story-11-5-live-dashboard-account.mjs'
- 'tests/release-readiness/**'
- 'docs/ops/**'
- '.env.example'
- 'package.json'
- '.github/workflows/story-10-4-full-stack-smoke-rehearsal.yml'
Comment thread
kyuoogle marked this conversation as resolved.
workflow_dispatch:

permissions:
contents: read

concurrency:
group: story-10-4-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref || github.run_id }}
cancel-in-progress: true

jobs:
full-stack-smoke-rehearsal:
name: story-10-4-full-stack-smoke-rehearsal
runs-on: ubuntu-latest
timeout-minutes: 90
env:
STORY_10_4_BUILD_ID: gha-${{ github.run_id }}-${{ github.run_attempt }}
COMPOSE_PROFILES: observability
VAULT_DEV_ROOT_TOKEN_ID: ci-vault-root-token
INTERNAL_SECRET_BOOTSTRAP: ci-bootstrap-secret
INTERNAL_SECRET: ci-runtime-secret
OBSERVABILITY_GRAFANA_ADMIN_USER: admin
OBSERVABILITY_GRAFANA_ADMIN_PASSWORD: admin
FEP_MARKETDATA_PROVIDER: REPLAY
SESSION_ISOLATION_BASE_URL: http://127.0.0.1:8080
SESSION_ISOLATION_PASSWORD: LiveVideo115!
SESSION_ISOLATION_SESSION_TIMEOUT_MS: "180000"
Comment thread
kyuoogle marked this conversation as resolved.
SESSION_ISOLATION_REQUEST_TIMEOUT_MS: "90000"
SESSION_ISOLATION_POLL_TIMEOUT_MS: "90000"
ROLLBACK_REHEARSAL_MODE: simulate
ROLLBACK_REHEARSAL_OPERATOR: github-actions
ROLLBACK_REHEARSAL_CHANGE_REF: GHA-${{ github.run_id }}
ROLLBACK_REHEARSAL_OWNER: release-platform-oncall
NODE_TLS_REJECT_UNAUTHORIZED: "0"
EDGE_BASE_URL: https://127.0.0.1
SMOKE_MANDATORY_API_BASE_URL: http://127.0.0.1:8080

steps:
- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
submodules: recursive
token: ${{ secrets.SUBMODULES_TOKEN || secrets.GITHUB_TOKEN }}

- name: Setup Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "20"

- name: Materialize rehearsal environment file
shell: bash
run: |
set -euo pipefail
cp .env.example .env
{
printf 'COMPOSE_PROFILES=%s\n' "${COMPOSE_PROFILES}"
printf 'VAULT_DEV_ROOT_TOKEN_ID=%s\n' "${VAULT_DEV_ROOT_TOKEN_ID}"
printf 'INTERNAL_SECRET_BOOTSTRAP=%s\n' "${INTERNAL_SECRET_BOOTSTRAP}"
printf 'INTERNAL_SECRET=%s\n' "${INTERNAL_SECRET}"
printf 'OBSERVABILITY_GRAFANA_ADMIN_USER=%s\n' "${OBSERVABILITY_GRAFANA_ADMIN_USER}"
printf 'OBSERVABILITY_GRAFANA_ADMIN_PASSWORD=%s\n' "${OBSERVABILITY_GRAFANA_ADMIN_PASSWORD}"
printf 'FEP_MARKETDATA_PROVIDER=%s\n' "${FEP_MARKETDATA_PROVIDER}"
} >> .env

- name: Validate release-readiness scripts
run: npm run lint:release-readiness

- name: Run release-readiness regression tests
run: npm run test:release-readiness

- name: Run full-stack smoke
id: smoke
continue-on-error: true
shell: bash
env:
SMOKE_BUILD_ID: ${{ env.STORY_10_4_BUILD_ID }}
SMOKE_STACK_READY_TIMEOUT_SECONDS: "300"
run: |
set -euo pipefail
bash ./scripts/release-readiness/run-full-stack-smoke.sh

- name: Run edge gateway validation
id: edge
if: ${{ always() && steps.smoke.outcome == 'success' }}
continue-on-error: true
shell: bash
env:
EDGE_VALIDATION_BUILD_ID: ${{ env.STORY_10_4_BUILD_ID }}
SKIP_COMPOSE_UP: "1"
run: |
set -euo pipefail
bash ./scripts/release-readiness/run-edge-gateway-validation.sh

Comment thread
kyuoogle marked this conversation as resolved.
- name: Run five-session isolation rehearsal
id: session_isolation
if: ${{ always() && steps.smoke.outcome == 'success' }}
continue-on-error: true
env:
SESSION_ISOLATION_BUILD_ID: ${{ env.STORY_10_4_BUILD_ID }}
run: node ./scripts/release-readiness/run-five-session-isolation.mjs

- name: Run rollback rehearsal
id: rollback
if: always()
continue-on-error: true
shell: bash
env:
ROLLBACK_REHEARSAL_BUILD_ID: ${{ env.STORY_10_4_BUILD_ID }}
run: |
set -euo pipefail
bash ./scripts/release-readiness/run-rollback-rehearsal.sh

- name: Assemble Story 10.4 evidence
id: assemble
if: always()
continue-on-error: true
env:
STORY_10_4_BUILD_ID: ${{ env.STORY_10_4_BUILD_ID }}
run: npm run assemble:story-10-4:evidence

- name: Upload Story 10.4 evidence
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
with:
name: story-10-4-full-stack-smoke-rehearsal-${{ github.run_id }}-${{ github.run_attempt }}
path: _bmad-output/test-artifacts/epic-10/${{ env.STORY_10_4_BUILD_ID }}/story-10-4/**
if-no-files-found: error
retention-days: 90

- name: Tear down rehearsal stack
if: always()
shell: bash
run: |
docker compose down -v --remove-orphans || true
rm -f .env

- name: Enforce Story 10.4 gate
if: always()
shell: bash
run: |
failures=0

if [[ "${{ steps.smoke.outcome }}" != "success" ]]; then
echo "Smoke step failed."
failures=1
fi

if [[ "${{ steps.smoke.outcome }}" == "success" && "${{ steps.edge.outcome }}" != "success" ]]; then
echo "Edge gateway validation failed."
failures=1
fi

if [[ "${{ steps.smoke.outcome }}" == "success" && "${{ steps.session_isolation.outcome }}" != "success" ]]; then
echo "Session isolation rehearsal failed."
failures=1
fi

if [[ "${{ steps.rollback.outcome }}" != "success" ]]; then
echo "Rollback rehearsal failed."
failures=1
fi

if [[ "${{ steps.assemble.outcome }}" != "success" ]]; then
echo "Story 10.4 evidence assembly failed."
failures=1
fi

exit "${failures}"
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,8 @@ docs/ops/redis-recovery/**/*.json
gradlew
gradlew.bat
*.sh
!scripts/observability/validate-observability-stack.sh
!scripts/infra-bootstrap/repair-service-databases.sh
!scripts/release-readiness/run-edge-gateway-validation.sh
!scripts/release-readiness/run-full-stack-smoke.sh
!scripts/release-readiness/run-rollback-rehearsal.sh
2 changes: 1 addition & 1 deletion BE
Submodule BE updated 54 files
+13 −2 .github/workflows/ci-quality-gate.yml
+4 −0 build.gradle
+217 −0 channel-service/build.gradle
+2 −0 channel-service/src/main/java/com/fix/channel/dto/response/AccountOrderHistoryResponse.java
+2 −0 channel-service/src/main/java/com/fix/channel/dto/response/AdminMonitoringFreshnessResponse.java
+35 −0 channel-service/src/main/java/com/fix/channel/entity/ManualRecoveryQueueEntry.java
+79 −1 channel-service/src/main/java/com/fix/channel/repository/ManualRecoveryQueueEntryRepository.java
+10 −1 channel-service/src/main/java/com/fix/channel/repository/OrderSessionRepository.java
+19 −2 channel-service/src/main/java/com/fix/channel/service/AdminOrderReplayService.java
+42 −1 channel-service/src/main/java/com/fix/channel/service/ManualRecoveryQueueService.java
+72 −58 channel-service/src/main/java/com/fix/channel/service/OrderExecutionService.java
+50 −18 channel-service/src/main/java/com/fix/channel/service/OrderSessionMonitoringMetrics.java
+28 −2 channel-service/src/main/java/com/fix/channel/service/OrderSessionPersistenceService.java
+53 −0 channel-service/src/main/java/db/migration/V24__add_manual_recovery_queue_resolution_columns.java
+27 −0 channel-service/src/test/java/com/fix/channel/contract/ChannelOpenApiCompatibilityTest.java
+40 −0 channel-service/src/test/java/com/fix/channel/entity/ManualRecoveryQueueEntryTest.java
+232 −0 channel-service/src/test/java/com/fix/channel/integration/AdminOrderReplayIntegrationTest.java
+3 −1 channel-service/src/test/java/com/fix/channel/integration/ChannelAuthSessionIntegrationTest.java
+139 −0 channel-service/src/test/java/com/fix/channel/integration/OrderSessionIntegrationTest.java
+360 −0 channel-service/src/test/java/com/fix/channel/perf/OrderExecuteLatencySmokeTest.java
+17 −1 channel-service/src/test/java/com/fix/channel/service/AdminOrderReplayServiceTest.java
+99 −5 channel-service/src/test/java/com/fix/channel/service/ManualRecoveryQueueServiceTest.java
+8 −1 channel-service/src/test/java/com/fix/channel/service/OrderExecutionServiceTest.java
+52 −11 channel-service/src/test/java/com/fix/channel/service/OrderSessionMonitoringMetricsTest.java
+170 −34 contracts/openapi/channel-service.json
+31 −31 contracts/openapi/corebank-service.json
+237 −0 corebank-service/build.gradle
+1 −1 corebank-service/src/main/java/com/fix/corebank/config/CorebankOpenApiDocumentationConfig.java
+3 −3 corebank-service/src/main/java/com/fix/corebank/security/InternalSecretFilter.java
+30 −1 .../resilienceScenario/java/com/fix/corebank/integration/CorebankSimulatorDrivenResilienceIntegrationTest.java
+100 −0 corebank-service/src/test/java/com/fix/corebank/contract/CorebankMatchingAntiRegressionTest.java
+118 −0 corebank-service/src/test/java/com/fix/corebank/contract/CorebankMatchingContractTest.java
+3 −3 corebank-service/src/test/java/com/fix/corebank/contract/CorebankOpenApiCompatibilityTest.java
+1 −1 corebank-service/src/test/java/com/fix/corebank/controller/CorebankInternalApiSkeletonTest.java
+322 −0 corebank-service/src/test/java/com/fix/corebank/integration/CorebankMatchingPersistenceIntegrationTest.java
+3 −1 corebank-service/src/test/java/com/fix/corebank/integration/CorebankOrderIdempotencyIntegrationTest.java
+3 −1 corebank-service/src/test/java/com/fix/corebank/integration/LedgerIntegrityIntegrationTest.java
+1 −1 corebank-service/src/test/java/com/fix/corebank/integration/LedgerIntegrityObservabilityIntegrationTest.java
+96 −36 corebank-service/src/test/java/com/fix/corebank/integration/PositionConcurrencyIntegrationTest.java
+3 −1 corebank-service/src/test/java/com/fix/corebank/integration/PositionLockContentionIntegrationTest.java
+6 −4 corebank-service/src/test/java/com/fix/corebank/security/InternalSecretFilterTest.java
+51 −77 corebank-service/src/test/java/com/fix/corebank/service/CorebankMatchingEngineTest.java
+40 −59 corebank-service/src/test/java/com/fix/corebank/service/MarketOrderSweepMatcherTest.java
+337 −0 corebank-service/src/test/java/com/fix/corebank/support/CorebankMatchingScenarioFixtures.java
+14 −0 docs/testing/epic-10-acceptance-matrix.md
+17 −0 docs/testing/epic-10-concurrency-performance-matrix.md
+84 −0 docs/testing/epic-10-concurrency-performance-observability.md
+15 −0 docs/testing/epic-10-resilience-drill-matrix.md
+61 −0 docs/testing/epic-10-resilience-evidence.md
+1 −1 fep-gateway/src/test/java/com/fix/fepgateway/controller/FepGatewayReplayTimelineContractTest.java
+4 −4 ...ateway/src/test/java/com/fix/fepgateway/dataplane/marketdata/replay/ReplayCursorPersistenceServiceTest.java
+217 −0 gradle/epic10-acceptance.gradle
+324 −0 gradle/epic10-concurrency-performance.gradle
+172 −0 gradle/epic10-resilience.gradle
1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,7 @@ services:
EDGE_TLS_KEY_PATH: /etc/nginx/certs/tls.key
CHANNEL_SERVICE_HOST: channel-service
CHANNEL_SERVICE_PORT: 8080
CHANNEL_SERVICE_HEALTH_PORT: 18080
COREBANK_SERVICE_HOST: corebank-service
COREBANK_SERVICE_PORT: 8081
FEP_GATEWAY_HOST: fep-gateway
Expand Down
3 changes: 2 additions & 1 deletion docker/nginx/docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ set -euo pipefail

: "${CHANNEL_SERVICE_HOST:=channel-service}"
: "${CHANNEL_SERVICE_PORT:=8080}"
: "${CHANNEL_SERVICE_HEALTH_PORT:=18080}"
: "${COREBANK_SERVICE_HOST:=corebank-service}"
: "${COREBANK_SERVICE_PORT:=8081}"
: "${FEP_GATEWAY_HOST:=fep-gateway}"
Expand All @@ -32,7 +33,7 @@ fi
touch "${DMZ_TEMP_DENYLIST_STATE_PATH}"
/usr/local/bin/dmz-temp-denylist.sh sweep >/dev/null 2>&1 || true

envsubst '${EDGE_SERVER_NAME} ${EDGE_TLS_CERT_PATH} ${EDGE_TLS_KEY_PATH} ${EDGE_TRUSTED_PROXY_CIDR_1} ${EDGE_TRUSTED_PROXY_CIDR_2} ${CHANNEL_SERVICE_HOST} ${CHANNEL_SERVICE_PORT} ${COREBANK_SERVICE_HOST} ${COREBANK_SERVICE_PORT} ${FEP_GATEWAY_HOST} ${FEP_GATEWAY_PORT} ${FEP_SIMULATOR_HOST} ${FEP_SIMULATOR_PORT}' \
envsubst '${EDGE_SERVER_NAME} ${EDGE_TLS_CERT_PATH} ${EDGE_TLS_KEY_PATH} ${EDGE_TRUSTED_PROXY_CIDR_1} ${EDGE_TRUSTED_PROXY_CIDR_2} ${CHANNEL_SERVICE_HOST} ${CHANNEL_SERVICE_PORT} ${CHANNEL_SERVICE_HEALTH_PORT} ${COREBANK_SERVICE_HOST} ${COREBANK_SERVICE_PORT} ${FEP_GATEWAY_HOST} ${FEP_GATEWAY_PORT} ${FEP_SIMULATOR_HOST} ${FEP_SIMULATOR_PORT}' \
< "${EDGE_NGINX_TEMPLATE}" \
> /etc/nginx/conf.d/fixyz-edge.conf

Expand Down
28 changes: 27 additions & 1 deletion docker/nginx/scripts/validate-edge-gateway.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
cd "${ROOT_DIR}"

BASE_URL="${EDGE_BASE_URL:-https://localhost}"
BASE_URL="${EDGE_BASE_URL:-https://127.0.0.1}"
COMPOSE_FILE="${EDGE_COMPOSE_FILE:-docker-compose.yml}"
TLS_HOST="${EDGE_TLS_HOST:-}"
TLS_PORT="${EDGE_TLS_PORT:-}"
Expand Down Expand Up @@ -55,6 +55,31 @@ fail() {
exit 1
}

wait_for_service_health() {
local service_name="$1"
local timeout_seconds="${2:-90}"
local started_at
local container_id=""
local health_status=""
started_at="$(date +%s)"

while true; do
container_id="$(docker compose -f "${COMPOSE_FILE}" ps -q "${service_name}" | tail -n1)"
if [[ -n "${container_id}" ]]; then
health_status="$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' "${container_id}")"
if [[ "${health_status}" == "healthy" || "${health_status}" == "running" ]]; then
return 0
fi
fi

if (( $(date +%s) - started_at >= timeout_seconds )); then
fail "Service ${service_name} did not become healthy within ${timeout_seconds}s after restart"
fi

sleep 2
done
}

curl_status() {
local route="$1"
local output_file="$2"
Expand Down Expand Up @@ -304,6 +329,7 @@ done

if [[ "${channel_service_was_running}" == "1" ]]; then
docker compose -f "${COMPOSE_FILE}" up -d channel-service >/dev/null
wait_for_service_health channel-service 90
fi
need_channel_service_restart=0
channel_service_was_running=0
Expand Down
7 changes: 6 additions & 1 deletion docker/nginx/templates/fixyz-edge.conf.template
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@ upstream channel_service {
keepalive 32;
}

upstream channel_service_health {
server ${CHANNEL_SERVICE_HOST}:${CHANNEL_SERVICE_HEALTH_PORT};
keepalive 8;
}

upstream corebank_service {
server ${COREBANK_SERVICE_HOST}:${COREBANK_SERVICE_PORT};
keepalive 32;
Expand Down Expand Up @@ -80,7 +85,7 @@ server {
if ($request_method != GET) {
return 404 '{"error":"EDGE_METHOD_NOT_ALLOWED","status":404,"request_id":"$request_id"}';
}
proxy_pass http://channel_service/actuator/health;
proxy_pass http://channel_service_health/actuator/health;
}

location = /health/corebank {
Expand Down
Loading
Loading