You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _data-prepper/pipelines/contains.md
+80-2Lines changed: 80 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,7 +18,7 @@ The function returns `true` if the substring specified in the second argument is
18
18
For example, if you want to check if the string `"abcd"` is contained within the value of a field named `message`, you can use the `contains()` function as follows:
19
19
20
20
```
21
-
contains('/message', 'abcd')
21
+
'contains(/message, "abcd")'
22
22
```
23
23
{% include copy.html %}
24
24
@@ -27,11 +27,89 @@ This call returns `true` if the field `message` contains the substring `abcd` or
27
27
Alternatively, you can use a literal string as the first argument:
28
28
29
29
```
30
-
contains('This is a test message', 'test')
30
+
'contains("This is a test message", "test")'
31
31
```
32
32
{% include copy.html %}
33
33
34
34
In this case, the function returns `true` because the substring `test` is present within the string `This is a test message`.
35
35
36
36
The `contains()` function performs a case-sensitive search.
37
37
{: .note}
38
+
39
+
## Example
40
+
41
+
The following pipeline uses the `contains()` function to add a Boolean flag `has_test` based on a substring in `/message` and filters out non-matching events, forwarding only messages that contain the string `ERROR` to OpenSearch:
42
+
43
+
```yaml
44
+
contains-demo-pipeline:
45
+
source:
46
+
http:
47
+
ssl: false
48
+
49
+
processor:
50
+
- add_entries:
51
+
entries:
52
+
- key: has_test
53
+
value_expression: contains(/message, "test")
54
+
- drop_events:
55
+
drop_when: not contains(/message, "ERROR")
56
+
57
+
sink:
58
+
- opensearch:
59
+
hosts: ["https://opensearch:9200"]
60
+
insecure: true
61
+
username: admin
62
+
password: admin_password
63
+
index_type: custom
64
+
index: demo-index-%{yyyy.MM.dd}
65
+
```
66
+
{% include copy.html %}
67
+
68
+
You can test the pipeline using the following command:
69
+
70
+
```bash
71
+
curl -sS -X POST "http://localhost:2021/log/ingest" \
72
+
-H "Content-Type: application/json" \
73
+
-d '[
74
+
{"message":"ok hello"},
75
+
{"message":"this has test but ok"},
76
+
{"message":"ERROR: something bad"},
77
+
{"message":"ERROR: unit test failed"}
78
+
]'
79
+
```
80
+
{% include copy.html %}
81
+
82
+
The documents stored in OpenSearch contain the following information:
0 commit comments