Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/vale-styles/Yugabyte/spelling-exceptions.txt
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,7 @@ downvoted
downvotes
Dpl
Dreamweaver
Dynatrace
Ebean
Ecto
Elasticsearch
Expand Down
4 changes: 2 additions & 2 deletions docs/content/stable/integrations/prisma.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,11 +84,11 @@ When finished, the following three files have created in the project directory,

## 2. Start the Prisma server

To start the Prisma server (in the Docker container) and launch the connected YugabyteDB database, navigate to the `prisma-yb` directory and then run the `docker-compose` command:
To start the Prisma server (in the Docker container) and launch the connected YugabyteDB database, navigate to the `prisma-yb` directory and then run the `docker compose` command:

```sh
cd prisma-yb
docker-compose up -d
docker compose up -d
```

You should now have a `prismagraphql/prisma` container running. You can check that by running `docker ps`.
Expand Down
32 changes: 14 additions & 18 deletions docs/content/stable/yugabyte-platform/alerts-monitoring/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ YugabyteDB Anywhere provides preconfigured alerts to notify you of potential pro

Use the Live Queries and Slow Queries dashboards to monitor and display current running queries and analyze statistics of past queries on your YugabyteDB universes.

Integrate with third-party tools such as Datadog, Dynatrace, and Splunk to export database metrics and audit logs for analysis.

{{<index/block>}}

{{<index/item
Expand All @@ -37,31 +39,31 @@ Use the Live Queries and Slow Queries dashboards to monitor and display current
title="Performance metrics"
body="View universe performance metrics."
href="anywhere-metrics/"
icon="/images/section_icons/explore/high_performance.png">}}
icon="fa-thin fa-chart">}}

{{<index/item
title="Live queries dashboard"
body="Monitor and display current running queries on your YugabyteDB universes."
body="Monitor and display current running queries."
href="live-queries-dashboard/"
icon="/images/section_icons/explore/monitoring.png">}}
icon="fa-thin fa-gauge-circle-bolt">}}

{{<index/item
title="Slow queries dashboard"
body="Analyze statistics and execution times of past queries on your YugabyteDB universes."
body="Analyze statistics and execution times of past queries."
href="slow-queries-dashboard/"
icon="/images/section_icons/explore/monitoring.png">}}
icon="fa-thin fa-turtle">}}

{{<index/item
title="Performance advisor"
body="Scan your database for potential optimizations."
href="performance-advisor/"
icon="/images/section_icons/manage/diagnostics.png">}}
icon="fa-thin fa-user-robot">}}

{{<index/item
title="Latency histogram and P99 latencies"
body="Use the Slow Queries dashboard to view latency histogram and P99 latency metrics."
body="View latency histogram and P99 latency metrics."
href="latency-histogram/"
icon="/images/section_icons/introduction/benefits.png">}}
icon="fa-thin fa-chart-simple">}}

{{<index/item
title="CDC observability"
Expand All @@ -70,21 +72,15 @@ Use the Live Queries and Slow Queries dashboards to monitor and display current
icon="fa-regular fa-arrows-spin">}}

{{<index/item
title="Export YSQL audit logs"
body="Export universe audit logs to third-party logging tools."
href="universe-logging/"
icon="/images/section_icons/explore/monitoring.png">}}

{{<index/item
title="Create export configurations"
body="Configure telemetry providers."
title="Export metrics and logs"
body="Export audit logs and node-level metrics to third-party tools."
href="anywhere-export-configuration/"
icon="/images/section_icons/explore/monitoring.png">}}
icon="fa-thin fa-chart-pie">}}

{{<index/item
title="Custom Prometheus"
body="Use your own Prometheus server to scrape metrics."
href="prometheus-custom/"
icon="/images/section_icons/introduction/benefits.png">}}
icon="fa-thin fa-fire">}}

{{</index/block>}}
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
---
title: Integrate with third-party tools in YugabyteDB Anywhere
headerTitle: Manage export configurations
linkTitle: Export configuration
linkTitle: Log and metrics export
description: Set up links to third-party tools in YugabyteDB Anywhere.
headcontent: Set up links to third-party tools
tags:
feature: early-access
menu:
stable_yugabyte-platform:
identifier: anywhere-export-configurations
Expand All @@ -14,43 +12,59 @@ menu:
type: docs
---

You can export universe logs to third-party tools for analysis and customization. Exporting data is a two-stage process:
You can export universe metrics and logs to third-party tools for analysis and customization.

1. Create an export configuration. A configuration defines the sign in credentials and settings for the tool that you want to export to.
1. Use the configuration to export logs from a universe. While the connection is active, logs are automatically streamed to the tool.
To export either metrics or logs from a universe:

Currently, you can export data to the following tools:
1. [Create an export configuration](#configure-integrations) for the integration you want to use. A configuration defines the sign in credentials and settings for the tool that you want to export to.

1. Using the configuration you created, connect your cluster.

- [Export metrics](../anywhere-metrics-export/)
- [Export logs](../universe-logging/)

- [Datadog](https://docs.datadoghq.com/)
- [Splunk](https://www.splunk.com/en_us/solutions/opentelemetry.html)
- [AWS CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html)
- [GCP Cloud Logging](https://cloud.google.com/logging/)
While the connection is active, metrics or logs are automatically streamed to the tool.

For information on how to export logs from a universe using an export configuration, refer to [Export logs](../universe-logging/).
To be able to export logs from Kubernetes universes, ensure the OpenTelemetry Operator is installed. Refer to [OpenTelemetry Operator for Kubernetes](https://opentelemetry.io/docs/platforms/kubernetes/operator/#getting-started) in the OpenTelemetry documentation. Metrics export is not supported on Kubernetes.

## Prerequisites
## Available integrations

Export configuration is {{<tags/feature/ea idea="792">}}. To enable export configuration management, set the **Enable DB Audit Logging** Global Configuration option (config key `yb.universe.audit_logging_enabled`) to true. Refer to [Manage runtime configuration settings](../../administer-yugabyte-platform/manage-runtime-config/). Note that only a Super Admin user can modify Global configuration settings. The flag can't be turned off if audit logging is enabled on a universe.
Currently, you can export data to the following tools:

| Integration | Log export | Metric export |
| :---------- | :--------- | :------------ |
| [Datadog](https://docs.datadoghq.com/) | Database audit logs | Yes |
| [Splunk](https://www.splunk.com/en_us/solutions/opentelemetry.html) | Database audit logs | |
| [AWS CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html) | Database audit logs | |
| [Google Cloud Logging](https://cloud.google.com/logging/) | Database audit logs | |
| [Dynatrace](#dynatrace) | | Yes |

## Best practices

- To limit performance impact and control costs, locate export configurations in a region close to your universe(s).
To limit performance impact and control costs, locate export configurations in a region close to your universe(s).

## Configure integrations
## Manage integrations

Create and manage export configurations on the **Integrations > Log** page.
Create and manage export configurations on the **Integrations > Logs and Metrics Export** page.

<!--![Export configurations](/images/yp/export-configurations.png)-->
The page lists the configured third-party integrations.

The page lists the configured and available third-party integrations.
To view details for a configuration, select it in the list.

### Manage integrations
To delete a configuration, click the three dots, and choose **Delete configuration**. You can't delete a configuration that is assigned to a universe.

You can add and delete export configurations for the following tools. You can't delete a configuration that is in use by a universe.
Note that you can't modify an existing configuration. If you need to change an configuration (for example, to replace or update an API key) for a particular tool, do the following:

1. Create a new configuration for the integration with the updated information.
1. Assign the new configuration to your universes.
1. Unassign the old configuration from universes.
1. Delete the old configuration.

## Configure integrations

{{< tabpane text=true >}}
You can add and delete export configurations for the following tools. You can't delete a configuration that is in use by a universe.

{{% tab header="Datadog" lang="datadog" %}}
### Datadog

The Datadog export configuration requires the following:

Expand All @@ -59,16 +73,14 @@ The Datadog export configuration requires the following:

To create an export configuration, do the following:

1. On the **Integrations** page, on the **Log** tab, click **Create Export Configuration**.
1. On the **Integrations** page, on the **Log & Metrics Export** tab, click **Add Configuration**.
1. Enter a name for the configuration.
1. Choose **Datadog**.
1. Enter your Datadog [API key](https://docs.datadoghq.com/account_management/api-app-keys/).
1. Choose the Datadog site to connect to, or choose Self-hosted and enter your URL.
1. Click **Create Configuration**.

{{% /tab %}}

{{% tab header="Splunk" lang="splunk" %}}
### Splunk

The Splunk export configuration requires the following:

Expand All @@ -77,17 +89,15 @@ The Splunk export configuration requires the following:

To create an export configuration, do the following:

1. On the **Integrations** page, on the **Log** tab, click **Create Export Configuration**.
1. On the **Integrations** page, on the **Log & Metrics Export** tab, click **Add Configuration**.
1. Enter a name for the configuration.
1. Choose **Splunk**.
1. Enter your Splunk [Access token](https://docs.splunk.com/observability/en/admin/authentication/authentication-tokens/org-tokens.html).
1. Enter the Endpoint URL.
1. Optionally, enter the Source, Source Type, and Index.
1. Click **Validate and Create Configuration**.

{{% /tab %}}

{{% tab header="AWS" lang="aws" %}}
### AWS

The AWS CloudWatch export configuration requires the following:

Expand All @@ -97,7 +107,7 @@ The AWS CloudWatch export configuration requires the following:

To create an export configuration, do the following:

1. On the **Integrations** page, on the **Log** tab, click **Create Export Configuration**.
1. On the **Integrations** page, on the **Log & Metrics Export** tab, click **Add Configuration**.
1. Enter a name for the configuration.
1. Choose **AWS CloudWatch**.
1. Enter your access key and secret access key.
Expand All @@ -107,35 +117,39 @@ To create an export configuration, do the following:
1. Optionally, provide an endpoint URL.
1. Click **Validate and Create Configuration**.

{{% /tab %}}

{{% tab header="GCP" lang="gcp" %}}
### Google Cloud Logging

The GCP Cloud Logging export configuration requires the following:
The Google Cloud Logging export configuration requires the following:

- Google Service Account with the `roles/logging.logWriter` role.
- The Service Account credentials JSON key. The credentials should be scoped to the project where the log group is located.

To create an export configuration, do the following:

1. On the **Integrations** page, on the **Log** tab, click **Create Export Configuration**.
1. On the **Integrations** page, on the **Log & Metrics Export** tab, click **Add Configuration**.
1. Enter a name for the configuration.
1. Choose **GCP Cloud Logging**.
1. Optionally, provide the project name.
1. Upload the JSON file containing your Google Cloud credentials.
1. Click **Validate and Create Configuration**.

{{% /tab %}}
### Dynatrace

The [Dynatrace](https://www.dynatrace.com) integration requires the following:

{{< /tabpane >}}
- Publically-accessible endpoint URL of your Dynatrace instance. The endpoint URL is the URL of your Dynatrace instance.
- [Dynatrace Access Token](https://docs.dynatrace.com/docs/manage/identity-access-management/access-tokens-and-oauth-clients/access-tokens#create-api-token). The access token needs to have ingest metrics, ingest logs, ingest OpenTelemetry traces, and read API tokens [scope](https://docs.dynatrace.com/docs/manage/identity-access-management/access-tokens-and-oauth-clients/access-tokens#scopes).

To view configuration details, select the configuration.
To create an export configuration, do the following:

To delete a configuration, click **Actions** and choose **Delete**.
1. On the **Integrations** page, on the **Log & Metrics Export** tab, click **Add Configuration**.
1. Enter a name for the configuration.
1. Choose **Dynatrace**.
1. Enter the Dynatrace Endpoint URL.
1. Enter your Dynatrace Access Token.
1. Click **Validate and Create Configuration**.

You can't modify an existing configuration. If you need to change the configuration (for example, to replace or update an API key) for a particular tool, do the following:
## Next steps

1. Create a new configuration for the provider with the updated information.
1. Assign the new configuration to your universes.
1. Unassign the old configuration from universes.
1. Delete the old configuration.
- [Export metrics from a universe](../anywhere-metrics-export/)
- [Export logs from a universe](../universe-logging/)
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
title: Export metrics from YugabyteDB Anywhere universes
headerTitle: Export metrics
linkTitle: Export metrics
description: Export universe metrics to third-party tools.
headcontent: Export universe metrics to third-party tools
tags:
feature: early-access
menu:
stable_yugabyte-platform:
identifier: anywhere-metrics-export
parent: anywhere-export-configurations
weight: 10
type: docs
---

You can export [universe metrics](../anywhere-metrics/) to third-party tools for analysis and customization.

## Prerequisites

- The Metrics export feature is {{<tags/feature/ea idea="441">}}. To enable the feature in YugabyteDB Anywhere, set the **Enable Metrics Export** Global Configuration option (config key `yb.universe.metrics_export_enabled`) to true. Refer to [Manage runtime configuration settings](../../administer-yugabyte-platform/manage-runtime-config/). Note that only a Super Admin user can modify Global configuration settings. The flag can't be turned off if metrics export is enabled on a universe.

- Create an export configuration. A configuration defines the sign in credentials and settings for the tool that you want to export metrics to. Refer to [Manage export configurations](../anywhere-export-configuration/).

## Limitations

Metrics export is not available for Kubernetes universes.

## Export universe metrics

To enable or modify metrics export for a universe, do the following:

1. On the universe **Metrics** tab, click the gear icon and choose **Export Metrics**.
1. Enable the **Export Metrics from this Universe** option.
1. Select the [export configuration](../anywhere-export-configuration/) for the tool you want to export to.
1. Click **Apply Changes**.

To remove metrics export from a universe, disable the **Export Metrics from this Universe** option.
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,12 @@ type: docs

Monitor performance metrics for your universe to ensure the universe configuration matches its performance requirements using the universe **Metrics** page.

{{< tip title="Export metrics" >}}

You can export universe metrics to third-party tools for analysis and customization. Refer to [Export metrics](../anywhere-metrics-export/).

{{< /tip >}}

The **Metrics** page displays graphs representing information on operations, latency, and other parameters accumulated over time. By examining specific metrics, you can diagnose and troubleshoot issues.

You access metrics by navigating to **Universes > Universe-Name > Metrics**, as shown in the following illustration:
Expand Down Expand Up @@ -153,7 +159,7 @@ Resource metrics should be considered on a per-node basis.
| --------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | --------------------------------- |
| Total YSQL Ops / sec | The number of various transaction control operations, such as BEGIN, COMMIT, ROLLBACK, as well as other operations through the YSQL API. | This metric is informational and should not be subject to alerting. You may consider this information while examining alerts on other metrics. | ![Total YSQL Ops / sec](/images/yp/metrics101.png) |
| YSQL Op Latency (Avg) | The average time taken by various transaction control operations, such as BEGIN, COMMIT, ROLLBACK, as well as other operations through the YSQL API. | This metric is informational and should not be subject to alerting. You may consider this information while examining alerts on other metrics. | ![YSQL Op Latency (Avg)](/images/yp/metrics102.png) |
| Catalog Cache Misses | During YSQL query processing, system catalog (pg_catalog) tables that live on the YB-Master are cached on the local YSQL backend. This metric counts the number of YSQL catalog cache misses, where the data had to be fetched from the YB-Master. Also broken down by pg_catalog table that triggered the misses. | This metric is informational and should not be subject to alerting. You may consider this information while examining alerts on other metrics. | ![Catalog Cache Misses](/images/yp/metrics114.png)
| Catalog Cache Misses | During YSQL query processing, system catalog (pg_catalog) tables that live on the YB-Master are cached on the local YSQL backend. This metric counts the number of YSQL catalog cache misses, where the data had to be fetched from the YB-Master. Also broken down by pg_catalog table that triggered the misses. | This metric is informational and should not be subject to alerting. You may consider this information while examining alerts on other metrics. | ![Catalog Cache Misses](/images/yp/metrics114.png) |

### YCQL advanced

Expand Down
Loading