Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ntp] Undefined behaviour with multiple instances #942

Open
masci opened this issue Dec 13, 2017 · 1 comment
Open

[ntp] Undefined behaviour with multiple instances #942

masci opened this issue Dec 13, 2017 · 1 comment
Labels
[deprecated] team/agent-core Deprecated. Use metrics-logs / shared-components labels instead..

Comments

@masci
Copy link
Contributor

masci commented Dec 13, 2017

Describe what happened:

If you list multiple instances in the config file of the ntp check, the check would be run once per instance with metrics and service checks posted without any tag, see https://github.com/DataDog/datadog-agent/blob/master/pkg/collector/corechecks/network/ntp.go#L146

Describe what you expected:

To be able to distinguish which value is coming from which NTP server.

@masci masci added the [deprecated] team/agent-core Deprecated. Use metrics-logs / shared-components labels instead.. label Dec 18, 2017
@ls339
Copy link
Contributor

ls339 commented Jan 3, 2018

Test
Configure 3 ntp instances in conf.d/ntp.d/check.yaml

init_config:

instances:
  - offset_threshold: 60
    host: 0.asia.pool.ntp.org

  - offset_threshold: 60
    host: 0.north-america.pool.ntp.org

  - offset_threshold: 60
    host: localhost # We expect this to fail  

Result
We can't distinguish between instances if more than one instance is configured in the ntp check. Metric ntp.offset and service check ntp.in_sync will be submitted once per instance run with only a host tag so we will not be able to differentiate between the metric value and the service check status.
It's not recommended to specify more than 1 instance in an ntp check because the service check ntp.in_sync may flap if one of the configured instances does not respond. In our test, we see that instance 2 has a status of 3 (unknown) causing our monitor to flap.

instance 0

Service Checks:
[
  {
    "check": "ntp.in_sync",
    "host_name": "ntpinvestigation",
    "timestamp": 1514998223,
    "status": 0,
    "message": "",
    "tags": null
  }
]

instance 1

Service Checks:
[
  {
    "check": "ntp.in_sync",
    "host_name": "ntpinvestigation",
    "timestamp": 1514998224,
    "status": 0,
    "message": "",
    "tags": null
  }
]

instance 2

Service Checks:
[
  {
    "check": "ntp.in_sync",
    "host_name": "ntpinvestigation",
    "timestamp": 1514998224,
    "status": 3,
    "message": "",
    "tags": null
  }
]

image 2018-01-03 at 12 03 28 pm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[deprecated] team/agent-core Deprecated. Use metrics-logs / shared-components labels instead..
Projects
None yet
Development

No branches or pull requests

2 participants