From 7ada195fe3980bec8dac016d8ab104db3e6f31c7 Mon Sep 17 00:00:00 2001 From: Pieter Lust Date: Thu, 7 May 2026 16:21:02 +0200 Subject: [PATCH 1/5] Add ForcedEndpoints documentation for MessageBrokerConfig --- ...Generating_BrokerGateway_client_secrets.md | 2 +- .../MessageBrokerConfig_ForcedEndpoints.md | 38 +++++++++++++++++++ dataminer/Administrator_guide/toc.yml | 2 + .../BrokerGateway_Migration.md | 5 +++ .../Procedures/Investigating_NATS_Issues.md | 1 + .../Connect_to_cloud_with_DMZ.md | 10 +++-- 6 files changed, 53 insertions(+), 5 deletions(-) create mode 100644 dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md diff --git a/dataminer/Administrator_guide/DataMiner_Systems/Generating_BrokerGateway_client_secrets.md b/dataminer/Administrator_guide/DataMiner_Systems/Generating_BrokerGateway_client_secrets.md index 5c15b3b65d..1983a18851 100644 --- a/dataminer/Administrator_guide/DataMiner_Systems/Generating_BrokerGateway_client_secrets.md +++ b/dataminer/Administrator_guide/DataMiner_Systems/Generating_BrokerGateway_client_secrets.md @@ -8,7 +8,7 @@ From DataMiner 10.5.0 [CU14]/10.6.0 [CU2]/10.6.5 onwards onwards, this is supported for DataMiner server processes. +> - For DxMs, check the release notes of the specific DxM to find out from which version this is supported. + +## Configuration + +To override the NATS endpoints for all processes on a server, add a `ForcedEndpoints` array to the `BrokerGatewayConfig` section in `C:\ProgramData\Skyline Communications\DataMiner\MessageBrokerConfig.json`: + +```json +{ + "BrokerGatewayConfig": { + "CredentialsUrl": "https:///BrokerGateway/api/natsconnection/getnatsconnectiondetails", + "APIKeyPath": "C:\\Program Files\\Skyline Communications\\DataMiner BrokerGateway\\appsettings.runtime.json", + "ForcedEndpoints": [ "custom-host-1:4222", "custom-host-2:4222" ] + } +} +``` + +Each entry in `ForcedEndpoints` must be a string in the format `"host:port"`. + +When `ForcedEndpoints` is set, the process will: + +- Still call `CredentialsUrl` to retrieve the NATS credentials. +- Use only the endpoints listed in `ForcedEndpoints` to connect to NATS, ignoring any endpoints returned by BrokerGateway. The order of endpoints determines which endpoint will be preferred to connect to and which to fall back to next. + +When `ForcedEndpoints` is absent or empty, normal behavior applies and the endpoints are resolved through BrokerGateway. diff --git a/dataminer/Administrator_guide/toc.yml b/dataminer/Administrator_guide/toc.yml index bd19605b25..bfd05a42c6 100644 --- a/dataminer/Administrator_guide/toc.yml +++ b/dataminer/Administrator_guide/toc.yml @@ -153,6 +153,8 @@ items: topicUid: Disabling_automatic_NATS_config - name: Generating BrokerGateway client secrets topicUid: Generating_BrokerGateway_client_secrets + - name: Configuring forced NATS endpoints + topicUid: MessageBrokerConfig_ForcedEndpoints - name: Running BPA tests topicUid: Running_BPA_tests items: diff --git a/dataminer/DataMiner_Tools/BrokerGateway_Migration.md b/dataminer/DataMiner_Tools/BrokerGateway_Migration.md index 565bd229d1..ab0e2360ff 100644 --- a/dataminer/DataMiner_Tools/BrokerGateway_Migration.md +++ b/dataminer/DataMiner_Tools/BrokerGateway_Migration.md @@ -175,6 +175,11 @@ A typical example of this file’s contents is shown below: } ``` + Optionally, a `ForcedEndpoints` array can be added to override the NATS endpoints provided by BrokerGateway. + This setting should only be used in combination with a server that does not run a DataMiner agent. + For example servers hosting a [DMZ setup for dataminer.services connectivity](xref:Connect_to_cloud_with_DMZ) or [Dashboard Gateway](xref:Dashboard_Gateway_installation). + For more information, see [Configuring forced NATS endpoints](xref:MessageBrokerConfig_ForcedEndpoints). + > [!NOTE] > The NATSMigration tool has a hard‑coded 10‑minute timeout for completing the *ResetCluster* operation. If for some reason the migration cannot be completed within 10 minutes, or if something goes wrong during the migration, all Agents will revert back to using the SLNet-managed NATS solution. diff --git a/dataminer/Troubleshooting/Procedures/Investigating_NATS_Issues.md b/dataminer/Troubleshooting/Procedures/Investigating_NATS_Issues.md index 840201b9bf..80bf784b00 100644 --- a/dataminer/Troubleshooting/Procedures/Investigating_NATS_Issues.md +++ b/dataminer/Troubleshooting/Procedures/Investigating_NATS_Issues.md @@ -164,6 +164,7 @@ This file configures how DataMiner processes connect to BrokerGateway to obtain - `CredentialsUrl` typically points to the local Agent (using loopback or FQDN). This is the default setting unless it has been manually changed. - If the HTTPS certificate CN/SAN does not match the hostname used in the URL, clients may fail with TLS validation errors. - *appsettings.runtime.json* must be present at the path specified in `APIKeyPath`. +- If you want to override the NATS endpoints returned by BrokerGateway, you can add a `ForcedEndpoints` array. See [Configuring forced NATS endpoints](xref:MessageBrokerConfig_ForcedEndpoints). **Example:** diff --git a/dataminer/dataminer_services/Connecting_to_cloud/Connect_to_cloud_with_DMZ.md b/dataminer/dataminer_services/Connecting_to_cloud/Connect_to_cloud_with_DMZ.md index 59f583df0c..e16abfc66b 100644 --- a/dataminer/dataminer_services/Connecting_to_cloud/Connect_to_cloud_with_DMZ.md +++ b/dataminer/dataminer_services/Connecting_to_cloud/Connect_to_cloud_with_DMZ.md @@ -72,7 +72,11 @@ From version 2.7.0 of the CloudGateway DxM onwards, you can connect a DMS to dat 1. If you are using the **BrokerGateway-managed** NATS solution: - 1. Copy a `C:\Program Files\Skyline Communications\DataMiner BrokerGateway\appsettings.runtime.json` file from a DataMiner node to the same location on the DMZ. + 1. Obtain an API key for the DMZ server: + + - From DataMiner 10.5.0 [CU14]/10.6.0 [CU2]/10.6.5 onwards, [generate a BrokerGateway client secret](xref:Generating_BrokerGateway_client_secrets) and place the client secret file on the DMZ server. Then set `APIKeyPath` to the path of that file. + + - In earlier DataMiner versions, copy `C:\Program Files\Skyline Communications\DataMiner BrokerGateway\appsettings.runtime.json` from a DataMiner node to the same location on the DMZ. Then set `APIKeyPath` to the path of the copied file. 1. On the DMZ, open `C:\ProgramData\Skyline Communications\DataMiner\MessagebrokerConfig.json`. @@ -82,15 +86,13 @@ From version 2.7.0 of the CloudGateway DxM onwards, you can connect a DMS to dat { "BrokerGatewayConfig": { "CredentialsUrl": "https://SERVER/BrokerGateway/api/natsconnection/getnatsconnectiondetails", - "APIKeyPath": "C:\\Program Files\\Skyline Communications\\DataMiner BrokerGateway\\appsettings.runtime.json" + "APIKeyPath": "" } } ``` 1. Set the `CredentialsUrl` to point to one of the servers in the **internal network**. - 1. Ensure the `APIKeyPath` points to the destination location of the copied `appsettings.runtime.json` file. - 1. If you are using the **SLNet-managed** NATS solution: 1. From a DataMiner node in the internal network, copy `C:\Skyline DataMiner\SLCloud.xml` to the same folder on the DMZ. From 628fc2cf7a9fde7c727f89652b6f113d6940e482 Mon Sep 17 00:00:00 2001 From: Pieter Lust <120721827+lustpieter@users.noreply.github.com> Date: Thu, 7 May 2026 16:32:25 +0200 Subject: [PATCH 2/5] update RN Updated note for DataMiner server process support version. --- .../DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md b/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md index d66d503765..bdab720b4f 100644 --- a/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md +++ b/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md @@ -11,7 +11,7 @@ This setting can be configured in `MessageBrokerConfig.json`. This is useful in scenarios where a process must reach NATS through a fixed set of endpoints — for example, when connecting through a proxy, a DMZ, or a load balancer. > [!NOTE] -> - From DataMiner 10.6.7 onwards, this is supported for DataMiner server processes. +> - From DataMiner 10.6.7 onwards, this is supported for DataMiner server processes. > - For DxMs, check the release notes of the specific DxM to find out from which version this is supported. ## Configuration From a0ac945d48eb024c1570fa9250079271ed6d03eb Mon Sep 17 00:00:00 2001 From: Pieter Lust <120721827+lustpieter@users.noreply.github.com> Date: Fri, 8 May 2026 09:29:40 +0200 Subject: [PATCH 3/5] Update ForcedEndpoints format description --- .../DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md b/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md index bdab720b4f..3d7b433cca 100644 --- a/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md +++ b/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md @@ -28,7 +28,7 @@ To override the NATS endpoints for all processes on a server, add a `ForcedEndpo } ``` -Each entry in `ForcedEndpoints` must be a string in the format `"host:port"`. +Each entry in `ForcedEndpoints` can be a string in the format `"host:port"` or `"host"`. When `ForcedEndpoints` is set, the process will: From 1bbfed8433cebde0fd53471587a16d6bbc27008d Mon Sep 17 00:00:00 2001 From: Pieter Lust Date: Mon, 11 May 2026 10:54:44 +0200 Subject: [PATCH 4/5] Address comments --- .../MessageBrokerConfig_ForcedEndpoints.md | 6 ++++-- dataminer/DataMiner_Tools/BrokerGateway_Migration.md | 1 + 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md b/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md index 3d7b433cca..0fe1036b6c 100644 --- a/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md +++ b/dataminer/Administrator_guide/DataMiner_Systems/MessageBrokerConfig_ForcedEndpoints.md @@ -23,7 +23,7 @@ To override the NATS endpoints for all processes on a server, add a `ForcedEndpo "BrokerGatewayConfig": { "CredentialsUrl": "https:///BrokerGateway/api/natsconnection/getnatsconnectiondetails", "APIKeyPath": "C:\\Program Files\\Skyline Communications\\DataMiner BrokerGateway\\appsettings.runtime.json", - "ForcedEndpoints": [ "custom-host-1:4222", "custom-host-2:4222" ] + "ForcedEndpoints": [ "custom-host-1:4222", "custom-host-2:4222", "custom-host-3" ] } } ``` @@ -33,6 +33,8 @@ Each entry in `ForcedEndpoints` can be a string in the format `"host:port"` or ` When `ForcedEndpoints` is set, the process will: - Still call `CredentialsUrl` to retrieve the NATS credentials. -- Use only the endpoints listed in `ForcedEndpoints` to connect to NATS, ignoring any endpoints returned by BrokerGateway. The order of endpoints determines which endpoint will be preferred to connect to and which to fall back to next. +- Use only the endpoints listed in `ForcedEndpoints` to connect to NATS, ignoring any endpoints returned by BrokerGateway. + - The order of endpoints determines the connection preference and fallback order. + - These endpoints do not need to be a subset of the known IPs of BrokerGateway. When `ForcedEndpoints` is absent or empty, normal behavior applies and the endpoints are resolved through BrokerGateway. diff --git a/dataminer/DataMiner_Tools/BrokerGateway_Migration.md b/dataminer/DataMiner_Tools/BrokerGateway_Migration.md index ab0e2360ff..89612e6ffa 100644 --- a/dataminer/DataMiner_Tools/BrokerGateway_Migration.md +++ b/dataminer/DataMiner_Tools/BrokerGateway_Migration.md @@ -178,6 +178,7 @@ A typical example of this file’s contents is shown below: Optionally, a `ForcedEndpoints` array can be added to override the NATS endpoints provided by BrokerGateway. This setting should only be used in combination with a server that does not run a DataMiner agent. For example servers hosting a [DMZ setup for dataminer.services connectivity](xref:Connect_to_cloud_with_DMZ) or [Dashboard Gateway](xref:Dashboard_Gateway_installation). + [Data Aggregator DxM](xref:Data_Aggregator_DxM) does not work in combination with `ForcedEndpoints`. For more information, see [Configuring forced NATS endpoints](xref:MessageBrokerConfig_ForcedEndpoints). > [!NOTE] From b648aefe94d6c5f487a486b39ae43315b2dffb43 Mon Sep 17 00:00:00 2001 From: Marieke Goethals Date: Wed, 20 May 2026 11:41:09 +0200 Subject: [PATCH 5/5] Update BrokerGateway_Migration.md --- dataminer/DataMiner_Tools/BrokerGateway_Migration.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/dataminer/DataMiner_Tools/BrokerGateway_Migration.md b/dataminer/DataMiner_Tools/BrokerGateway_Migration.md index 89612e6ffa..a5f946ca7a 100644 --- a/dataminer/DataMiner_Tools/BrokerGateway_Migration.md +++ b/dataminer/DataMiner_Tools/BrokerGateway_Migration.md @@ -164,7 +164,7 @@ The following actions will be executed automatically during the migration, in th This file is used when initializing default sessions for DataMiner processes using the NATS communication channel. During system migration, the file is automatically overwritten to include the correct BrokerGateway URL and the path to the associated API key. -A typical example of this file’s contents is shown below: + A typical example of this file’s contents is shown below: ```json { @@ -175,11 +175,7 @@ A typical example of this file’s contents is shown below: } ``` - Optionally, a `ForcedEndpoints` array can be added to override the NATS endpoints provided by BrokerGateway. - This setting should only be used in combination with a server that does not run a DataMiner agent. - For example servers hosting a [DMZ setup for dataminer.services connectivity](xref:Connect_to_cloud_with_DMZ) or [Dashboard Gateway](xref:Dashboard_Gateway_installation). - [Data Aggregator DxM](xref:Data_Aggregator_DxM) does not work in combination with `ForcedEndpoints`. - For more information, see [Configuring forced NATS endpoints](xref:MessageBrokerConfig_ForcedEndpoints). + Optionally, a `ForcedEndpoints` array can be added to override the NATS endpoints provided by BrokerGateway. This setting should only be used in combination with a server that does not run a DataMiner Agent, for example, servers hosting a [DMZ setup for dataminer.services connectivity](xref:Connect_to_cloud_with_DMZ) or [Dashboard Gateway](xref:Dashboard_Gateway_installation). The [Data Aggregator DxM](xref:Data_Aggregator_DxM) does not work in combination with `ForcedEndpoints`. For more information, see [Configuring forced NATS endpoints](xref:MessageBrokerConfig_ForcedEndpoints). > [!NOTE] > The NATSMigration tool has a hard‑coded 10‑minute timeout for completing the *ResetCluster* operation. If for some reason the migration cannot be completed within 10 minutes, or if something goes wrong during the migration, all Agents will revert back to using the SLNet-managed NATS solution.