Skip to content

[MOSIP-43841] Changes related to alerting setup via rapid deployment. #117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Mohanraj209 wants to merge 8 commits into
base: develop
Choose a base branch
from
+54 −27
Open

[MOSIP-43841] Changes related to alerting setup via rapid deployment. #117

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ec41f23
[MOSIP-43841] Updated alerting-setup.sh
Mohanraj209 Nov 27, 2025
a51c8a7
[MOSIP-43841] Updated alertmanager.yaml
Mohanraj209 Nov 27, 2025
05244d3
[MOSIP-43841] Updated patch-cluster-name.yaml
Mohanraj209 Nov 27, 2025
91a16d9
[MOSIP-43841] Updated prereq-dsf.yaml
Mohanraj209 Nov 27, 2025
ea6b91b
Merge branch 'develop' of https://github.com/mosip/infra into develop
Jan 21, 2026
29a166f
[MOSIP-43841] Alerting issues related to rapid deployment fixed.
Mohanraj209 Jan 21, 2026
c53ae3b
[MOSIP-43841] Alerting issues related to rapid deployment fixed.
Mohanraj209 Jan 21, 2026
d18d1fb
[MOSIP-43841] Alerting issues related to rapid deployment fixed.
Mohanraj209 Jan 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions Helmsman/dsf/prereq-dsf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,23 +33,23 @@ apps:
timeout: 900
priority: -6
hooks:
postInstall: "$WORKDIR/hooks/install-istio-and-httpbin.sh sandbox.xyz.net helmsman"
postUpgrade: "$WORKDIR/hooks/install-istio-and-httpbin.sh sandbox.xyz.net helmsman"
postInstall: "$WORKDIR/hooks/install-istio-and-httpbin.sh <sandbox.xyz.net> <version>"
postUpgrade: "$WORKDIR/hooks/install-istio-and-httpbin.sh <sandbox.xyz.net> <version>"

rancher-monitoring:
namespace: cattle-monitoring-system
enabled: false
version: 103.1.0+up45.31.1
chart: mosip/rancher-monitoring
set:
grafana.global.cattle.clusterId: "c-m-pbrcfglw"
global.cattle.clusterId: "c-m-pbrcfglw"
grafana.global.cattle.clusterId: "<cluster-id>"
global.cattle.clusterId: "<cluster-id>"
wait: true
valuesFile: "$WORKDIR/utils/monitoring_values.yaml"
valuesFile: "$WORKDIR/utils/monitoring_values.yaml"
priority: -5
timeout: 600
hooks:
postInstall: "$WORKDIR/hooks/alerting-setup.sh"
postInstall: "$WORKDIR/hooks/alerting-setup.sh <slack-channel-name> <slack-api-url> <env-name>"

elasticsearch:
namespace: cattle-logging-system
Expand Down Expand Up @@ -85,7 +85,7 @@ apps:
version: 0.1.0
chart: $WORKDIR/utils/istio-addons/logging-istio-addons-0.1.0.tgz
set:
kibanaHost: "kibana.sandbox.xyz.net"
kibanaHost: "<kibana.sandbox.xyz.net>"
installName: "elasticsearch-kibana"
wait: true
priority: -3
Expand Down
6 changes: 3 additions & 3 deletions Helmsman/hooks/alerting-setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ NS=cattle-monitoring-system
function installing_alerting() {

# Define the Slack channel, Slack_api_url and Cluster name dynamically
SLACK_CHANNEL="soil"
SLACK_API_URL="https://hooks.slack.com/services/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"
ENV_NAME="soil"
SLACK_CHANNEL="$1"
SLACK_API_URL="$2"
ENV_NAME="$3"

ALERTMANAGER_FILE="$WORKDIR/utils/alerting/alertmanager.yaml"
PATCH_CLUSTER_NAME_FILE="$WORKDIR/utils/alerting/patch-cluster-name.yaml"
Expand Down
4 changes: 2 additions & 2 deletions Helmsman/utils/alerting/alertmanager.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
global:
resolve_timeout: 5m
slack_api_url: https://hooks.slack.com/services/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
slack_api_url: <YOUR-SLACK-API-URL>
route:
group_by:
- namespace
Expand Down Expand Up @@ -104,7 +104,7 @@ receivers:
- name: 'null'
- name: 'slack'
slack_configs:
- channel: 'soil'
- channel: '<YOUR-CHANNEL-HERE>'
send_resolved: true
title: '[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing |
len }}{{ end }}] Monitoring Event Notification'
Expand Down
2 changes: 1 addition & 1 deletion Helmsman/utils/alerting/patch-cluster-name.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
spec:
externalLabels:
cluster: soil
cluster: <YOUR-CLUSTER-NAME-HERE>
55 changes: 41 additions & 14 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -774,8 +774,24 @@ To regenerate import URL if needed:
global.cattle.clusterId: "c-m-pbrcfglw" # ← REPLACE THIS
```

- **Alerting Configuration:**
Alerting is part of cluster monitoring, where alert notifications are sent to the configured email or Slack channel.
- `<slack-channel-name>` → Slack channel name configured for alert notifications.
- `<slack-api-url>` → Slack API URL configured for alert notifications.
- `<env-name>` → provide the cluster name.

> **Note:**
> - Create a Slack incoming webhook: [Slack incoming webhooks guide](https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks/)
> - Create a Slack app for your environment from the above URL.
> - After creating the app, select `Incoming webhooks` from the **Features** section.
> - Activate Incoming webhooks.
> - Select `Add New Webhook To Workspace` and choose a Slack channel where alerts should be notified.
> - The incoming webhook URL will be created.
> - Update `slack_api_url`, `channel`, and `env-name` in the `rancher-monitoring` section in `prereq-dsf.yaml`.


- **Domain Validation (Double-check):**
- `<sandbox>` → your cluster name (e.g., `soil38`)
- `<env-name>` → your cluster name (e.g., `soil38`)
- `sandbox.xyz.net` → your domain name (e.g., `soil38.mosip.net`)
- **Why?** Every service needs to know its web address
- **Chart Versions:** Verify and update to latest stable versions
Expand All @@ -785,9 +801,10 @@ To regenerate import URL if needed:

> **Note:** Maintain consistency with your Terraform configuration:
>
> - `<sandbox>` should match `cluster_name` in `aws.tfvars`
> - `sandbox.xyz.net` should match `cluster_env_domain` in `aws.tfvars`
> - These MUST be identical or deployment will fail!
> - `<env-name>` should match `cluster_name` in `aws.tfvars`
> - `<sandbox.xyz.net>` should match `cluster_env_domain` in `aws.tfvars`.
> - These above variables MUST be identical or deployment will fail because the same domain is being mapped in the route-53 service in aws.
> - If the cluster is created manually instead of using terraform scripts then user can provide the `values` for above two variables as per his requirement, no need to match variables in `aws.tfvars`.

```yaml
# Configure monitoring, Istio, logging
Expand Down Expand Up @@ -815,9 +832,10 @@ To regenerate import URL if needed:
- **PostgreSQL Configuration:** Match with Terraform `enable_postgresql_setup` setting

> **Note:** Maintain consistency with your Terraform configuration:
>
> - `<sandbox>` should match `cluster_name` in `aws.tfvars`
> - `sandbox.xyz.net` should match `cluster_env_domain` in `aws.tfvars`
> - `<sandbox.xyz.net>` should match `cluster_env_domain` in `aws.tfvars`.
> - These above variables MUST be identical or deployment will fail because the same domain is being mapped in the route-53 service in aws.
> - If the cluster is created manually instead of using terraform scripts then user can provide the `values` for above two variables as per his requirement, no need to match variables in `aws.tfvars`.

- **Configure reCAPTCHA keys:**

Expand Down Expand Up @@ -870,8 +888,10 @@ To regenerate import URL if needed:
**Critical Updates Required:**

- **Domain Validation (Double-check):**
- `<sandbox>` → your cluster name (e.g., `soil`)
- `sandbox.xyz.net` → your domain name (e.g., `soil.mosip.net`)
> - `<sandbox>` should match `cluster_name` in `aws.tfvars`
> - `<sandbox.xyz.net>` should match `cluster_env_domain` in `aws.tfvars`.
> - These above variables MUST be identical or deployment will fail because the same domain is being mapped in the route-53 service in aws.
> - If the cluster is created manually instead of using terraform scripts then user can provide the `values` for above two variables as per his requirement, no need to match variables in `aws.tfvars`.
- **Chart Versions:** Update MOSIP service chart versions to compatible releases
- **Database Branch:** Ensure correct MOSIP DB scripts branch matches deployment version
- **Service Dependencies:** Verify all required external services are properly configured
Expand All @@ -880,7 +900,9 @@ To regenerate import URL if needed:
> **Note:** Maintain consistency with your Terraform configuration:
>
> - `<sandbox>` should match `cluster_name` in `aws.tfvars`
> - `sandbox.xyz.net` should match `cluster_env_domain` in `aws.tfvars`
> - `<sandbox.xyz.net>` should match `cluster_env_domain` in `aws.tfvars`.
> - These above variables MUST be identical or deployment will fail because the same domain is being mapped in the route-53 service in aws.
> - If the cluster is created manually instead of using terraform scripts then user can provide the `values` for above two variables as per his requirement, no need to match variables in `aws.tfvars`.

```yaml
# Configure MOSIP services
Expand All @@ -898,8 +920,10 @@ To regenerate import URL if needed:
**Critical Updates Required:**

- **Domain Validation (Double-check):**
- `<sandbox>` → your cluster name (e.g., `soil`)
- `sandbox.xyz.net` → your domain name (e.g., `soil.mosip.net`)
> - `<sandbox>` should match `cluster_name` in `aws.tfvars`
> - `<sandbox.xyz.net>` should match `cluster_env_domain` in `aws.tfvars`.
> - These above variables MUST be identical or deployment will fail because the same domain is being mapped in the route-53 service in aws.
> - If the cluster is created manually instead of using terraform scripts then user can provide the `values` for above two variables as per his requirement, no need to match variables in `aws.tfvars`.
- **Test Chart Versions:** Update test rig chart versions to match MOSIP service versions
- **Database Branch:** Ensure test DB scripts use correct branch
- **Test Configuration:** Update test endpoints, API versions, and test data paths
Expand All @@ -908,9 +932,10 @@ To regenerate import URL if needed:
> **Critical Validation Checklist for All DSF Files:**
>
> **Domain Configuration (Validate Twice):**
>
> - `<sandbox>` → your cluster name (e.g., `soil`)
> - `sandbox.xyz.net` → your domain name (e.g., `soil.mosip.net`)
> - `<sandbox>` should match `cluster_name` in `aws.tfvars`
> - `<sandbox.xyz.net>` should match `cluster_env_domain` in `aws.tfvars`.
> - These above variables MUST be identical or deployment will fail because the same domain is being mapped in the route-53 service in aws.
> - If the cluster is created manually instead of using terraform scripts then user can provide the `values` for above two variables as per his requirement, no need to match variables in `aws.tfvars`.
> - Verify domain DNS resolution is working
> - Ensure SSL certificate coverage for all subdomains
>
Expand All @@ -924,6 +949,8 @@ To regenerate import URL if needed:
>
> - `<sandbox>` must match `cluster_name` in `terraform/implementations/aws/infra/aws.tfvars`
> - `sandbox.xyz.net` must match `cluster_env_domain` in `terraform/implementations/aws/infra/aws.tfvars`
> - These above variables MUST be identical or deployment will fail because the same domain is being mapped in the route-53 service in aws.
> - If the cluster is created manually instead of using terraform scripts then user can provide the `values` for above two variables as per his requirement, no need to match variables in `aws.tfvars`.
> - PostgreSQL settings must align with `enable_postgresql_setup` in Terraform configuration
>
> **Environment-Specific Updates:**
Expand Down