Skip to content

Commit 5c91d4f

Browse files
create integration pages for windows and linux (#140)
* create integration pages for windows and linux * update the Linux and Windows integration pages * update the Linux and Windows integration pages
1 parent 6ff7969 commit 5c91d4f

File tree

5 files changed

+396
-1
lines changed

5 files changed

+396
-1
lines changed

docs/images/linux-integration.png

295 KB
Loading
356 KB
Loading

docs/integration/.pages

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,4 +6,7 @@ nav:
66
- Database: database
77
- Servers: servers
88
- DevOps : devops
9-
- Application Platform: application-platform
9+
- Linux: linux.md
10+
- Windows: windows.md
11+
- Application Platform: application-platform
12+

docs/integration/linux.md

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
---
2+
title: Linux Monitoring Integration with OpenObserve
3+
description: Learn how to integrate Linux systems with OpenObserve to collect and analyze system logs and performance metrics using the OpenObserve Collector or OpenTelemetry Collector.
4+
---
5+
# Integration with Linux
6+
7+
This guide explains how to integrate Linux systems with OpenObserve to collect and analyze system logs and performance metrics.
8+
9+
## Overview
10+
11+
Linux systems generate two primary types of monitoring data: system logs and performance metrics. Collecting and analyzing these data is essential for maintaining system health, performance, and security.
12+
13+
---
14+
15+
## Installation Options
16+
17+
OpenObserve provides two approaches to monitor Linux systems:
18+
19+
- **OpenObserve Collector**: A simple, one-command installation recommended for most users.
20+
- **OpenTelemetry Collector**: A flexible option for advanced or customized environments.
21+
22+
---
23+
24+
## Steps to integrate
25+
26+
=== "Using the OpenObserve Collector (Recommended)"
27+
28+
**Prerequisites**
29+
30+
- A Linux machine such as Ubuntu, Debian, CentOS, or RHEL
31+
- Root or sudo access to install and configure services
32+
- Access to an OpenObserve instance, either [cloud](https://openobserve.ai/docs/getting-started/#option-1-openobserve-cloud-setup) or [self-hosted](https://openobserve.ai/docs/getting-started/#option-2-self-hosted-installation)
33+
34+
**Steps**
35+
36+
1. Log in to your OpenObserve instance.
37+
2. Navigate to **Data Sources** > **Recommended** > **Linux**.
38+
3. Copy the provided installation command. The command includes your endpoint and API key.
39+
4. Run the command in your terminal:
40+
```
41+
curl -O https://raw.githubusercontent.com/openobserve/agents/main/linux/install.sh && chmod +x install.sh && sudo ./install.sh https://your-openobserve-instance.com/api/default/ YOUR_API_KEY
42+
```
43+
![linux integration](../images/linux-integration.png)
44+
45+
46+
What the OpenObserve Collector Does:
47+
48+
- Installs as a systemd service
49+
- Collects system logs from journald or syslog
50+
- Collects host metrics including CPU, memory, disk, network, and process-level usage
51+
- Forwards all collected data to OpenObserve
52+
53+
=== "Using the OpenTelemetry Collector"
54+
55+
**Steps**
56+
57+
1. Set Up the OpenTelemetry Collector
58+
Check for the latest version of the OpenTelemetry Collector on the official [releases page](https://github.com/open-telemetry/opentelemetry-collector-releases/releases).
59+
```bash linenums="1"
60+
sudo mkdir -p /opt/otel-collector
61+
cd /opt/otel-collector
62+
curl -L https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.115.0/otelcol-contrib_0.115.0_linux_amd64.tar.gz -o otelcol-contrib.tar.gz
63+
tar -xzf otelcol-contrib.tar.gz
64+
```
65+
2. Configure the OpenTelemetry Collector
66+
Create a file named `config.yaml` in the `/opt/otel-collector` directory with the following content:
67+
```yaml linenums="1"
68+
receivers:
69+
filelog:
70+
include:
71+
- /var/log/syslog
72+
- /var/log/auth.log
73+
- /var/log/kern.log
74+
- /var/log/messages
75+
- /var/log/secure
76+
start_at: end
77+
include_file_path: true
78+
include_file_name: true
79+
operators:
80+
- type: regex_parser
81+
regex: '^(?P<time>[A-Z][a-z]{2}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})\s+(?P<host>[^\s]+)\s+(?P<service>[^\s\[]+)(\[(?P<pid>\d+)\])?:\s+(?P<message>.*)$'
82+
timestamp:
83+
parse_from: time
84+
layout: Jan 02 15:04:05
85+
86+
hostmetrics:
87+
collection_interval: 30s
88+
scrapers:
89+
cpu:
90+
metrics:
91+
system.cpu.utilization:
92+
enabled: true
93+
memory:
94+
metrics:
95+
system.memory.utilization:
96+
enabled: true
97+
disk:
98+
filesystem:
99+
network:
100+
load:
101+
paging:
102+
process:
103+
mute_process_name_error: true
104+
metrics:
105+
process.cpu.utilization:
106+
enabled: true
107+
process.memory.utilization:
108+
enabled: true
109+
110+
processors:
111+
batch:
112+
send_batch_size: 1024
113+
timeout: 10s
114+
resourcedetection:
115+
detectors: [system, env]
116+
system:
117+
hostname_sources: ["os"]
118+
119+
exporters:
120+
otlphttp/openobserve:
121+
endpoint: "https://your-openobserve-instance.com/api/default"
122+
headers:
123+
Authorization: "Basic YOUR_API_KEY"
124+
stream-name: "linux-logs"
125+
debug:
126+
verbosity: detailed
127+
128+
service:
129+
pipelines:
130+
logs:
131+
receivers: [filelog]
132+
processors: [resourcedetection, batch]
133+
exporters: [otlphttp/openobserve, debug]
134+
metrics:
135+
receivers: [hostmetrics]
136+
processors: [resourcedetection, batch]
137+
exporters: [otlphttp/openobserve, debug]
138+
telemetry:
139+
logs:
140+
level: "info"
141+
```
142+
Replace `https://your-openobserve-instance.com/api/default` with your OpenObserve endpoint and `YOUR_API_KEY` with your actual API key.
143+
3. Run the Collector
144+
145+
```bash
146+
sudo ./otelcol-contrib --config config.yaml
147+
```
148+
149+
---
150+
151+
## Advanced Configuration Options
152+
Here are some essential configurations to enhance your Linux monitoring:
153+
154+
??? "Security-Focused Monitoring"
155+
For enhanced security monitoring:
156+
```bash linenums="1"
157+
filelog:
158+
include:
159+
- /var/log/auth.log
160+
- /var/log/secure
161+
- /var/log/audit/audit.log
162+
operators:
163+
# For SSH authentication events
164+
- type: router
165+
routes:
166+
- expr: 'includes(file.name, "auth.log") or includes(file.name, "secure")'
167+
output: auth-parser
168+
- type: regex_parser
169+
id: auth-parser
170+
regex: '(?P<time>[A-Z][a-z]{2}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})\s+(?P<host>[^\s]+)\s+sshd\[(?P<pid>\d+)\]:\s+(?P<message>.*)'
171+
timestamp:
172+
parse_from: time
173+
layout: Jan 02 15:04:05
174+
```
175+
This configuration focuses on authentication logs to help detect unauthorized access attempts and potential security breaches.
176+
177+
??? "Using Journald Receiver"
178+
If you prefer to collect logs directly from journald instead of log files:
179+
```bash linenums="1"
180+
journald:
181+
units:
182+
- ssh.service
183+
- systemd-logind.service
184+
priority: info
185+
```
186+
The journald receiver requires systemd and only works on Linux systems using systemd as their init system.
187+
188+
## Troubleshooting
189+
190+
??? "OpenObserve Collector Installation Issues"
191+
- **Permission Denied**: Run the installation command with `sudo`.
192+
- **Service Not Starting**: Check status with `sudo systemctl status openobserve-agent` and look for error messages.
193+
- **Network Issues**: Verify connectivity with `curl -v https://your-openobserve-instance.com`.
194+
195+
??? "Log Collection Issues"
196+
- **Missing Logs**: Check file permissions and ensure the OpenObserve Collector has read access to log files.
197+
- **Journald Access**: For systemd-based systems, ensure the OpenObserve Collector has proper journal access.
198+
- **Log Format Problems**: If logs appear malformed, check timezone settings and log formats.
199+
200+
??? "Metrics Collection Issues"
201+
- **Missing Metrics**: Verify the OpenObserve Collector has permissions to access system metrics.
202+
- **Performance Impact**: If the OpenObserve Collector uses too many resources, adjust collection intervals.
203+
204+
??? "OpenTelemetry Collector Issues"
205+
- **Configuration Errors**: Validate your config.yaml syntax and check OpenTelemetry Collector logs.
206+
- **Export Failures**: Verify your endpoint URL and API key are correct.
207+
- **Resource Constraints**: Adjust batch settings if the OpenTelemetry Collector terminates unexpectedly.
208+

0 commit comments

Comments
 (0)