Skip to content

zscaler_zpa: fix handling of multiple remote IPs, and event categorisation #13755

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions packages/zscaler_zpa/changelog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,12 @@
# newer versions go on top
- version: "1.22.2"
changes:
- description: Fix handling of remote IP lists in audit data stream.
type: bugfix
link: https://github.com/elastic/integrations/pull/13755
- description: Fix ECS event type, category and outcome mapping of audit events.
type: bugfix
link: https://github.com/elastic/integrations/pull/13755
- version: "1.22.1"
changes:
- description: Do not set `error.message` for expected behavior related to Zscaler `Host` field.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
{"ModifiedTime":"2021-11-17T04:29:38.000Z","CreationTime":"2021-11-17T04:29:38.000Z","ModifiedBy":12345678901234567,"RequestID":"11111111-1111-1111-1111-111111111111","SessionID":"1idn23nlfm2q1txa5h3r4mep6","AuditOldValue":"","AuditNewValue":"{\"id\":\"72058340288495701\",\"name\":\"Some-Name\",\"domainOrIpAddress\":\"81.2.69.144\",\"description\":\"This is a description field\",\"enabled\":\"true\"}","AuditOperationType":"Create","ObjectType":"Server","ObjectName":"Some-Name","ObjectID":12345678901234567,"CustomerID":98765432109876543,"User":"[email protected]","ClientAuditUpdate":0}
{"ModifiedTime":"","CreationTime":"2021-11-17T04:29:38.000Z","ModifiedBy":12345678901234567,"RequestID":"11111111-1111-1111-1111-111111111111","SessionID":"1idn23nlfm2q1txa5h3r4mep6","AuditOldValue":"","AuditNewValue":"{\"id\":\"72058340288495701\",\"name\":\"Some-Name\",\"domainOrIpAddress\":\"example.com\",\"description\":\"This is a description field\",\"enabled\":\"true\"}","AuditOperationType":"Create","ObjectType":"Server","ObjectName":"Some-Name","ObjectID":12345678901234567,"CustomerID":98765432109876543,"User":"[email protected]","ClientAuditUpdate":0}
{"ModifiedTime":"","CreationTime":"2025-04-30T16:23:40.000Z","ModifiedBy":288263728720249833,"RequestID":"12d6eccc-718c-4657-b267-83cc1c3f35f6","SessionID":"1samau4fwi7xbsf3317mkd5vz","AuditOldValue":"","AuditNewValue":"{\"loginAttempt\":\"2025-04-30 16:23:40 UTC\",\"remoteIP\":\"81.2.69.142, 81.2.69.144\"}","AuditOperationType":"Sign In","ObjectType":"Authentication","ObjectName":"xxxx","ObjectID":"xxxxx","CustomerID":"xxxxx","User":"xxxx","ClientAuditUpdate":1}
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"id": "11111111-1111-1111-1111-111111111111",
"kind": "event",
"original": "{\"ModifiedTime\":\"2021-11-17T04:29:38.000Z\",\"CreationTime\":\"2021-11-17T04:29:38.000Z\",\"ModifiedBy\":12345678901234567,\"RequestID\":\"11111111-1111-1111-1111-111111111111\",\"SessionID\":\"1idn23nlfm2q1txa5h3r4mep6\",\"AuditOldValue\":\"\",\"AuditNewValue\":\"{\\\"id\\\":\\\"72058340288495701\\\",\\\"name\\\":\\\"Some-Name\\\",\\\"domainOrIpAddress\\\":\\\"81.2.69.144\\\",\\\"description\\\":\\\"This is a description field\\\",\\\"enabled\\\":\\\"true\\\"}\",\"AuditOperationType\":\"Create\",\"ObjectType\":\"Server\",\"ObjectName\":\"Some-Name\",\"ObjectID\":12345678901234567,\"CustomerID\":98765432109876543,\"User\":\"[email protected]\",\"ClientAuditUpdate\":0}",
"outcome": "success",
"type": [
"creation"
]
Expand Down Expand Up @@ -77,6 +78,7 @@
"id": "11111111-1111-1111-1111-111111111111",
"kind": "event",
"original": "{\"ModifiedTime\":\"\",\"CreationTime\":\"2021-11-17T04:29:38.000Z\",\"ModifiedBy\":12345678901234567,\"RequestID\":\"11111111-1111-1111-1111-111111111111\",\"SessionID\":\"1idn23nlfm2q1txa5h3r4mep6\",\"AuditOldValue\":\"\",\"AuditNewValue\":\"{\\\"id\\\":\\\"72058340288495701\\\",\\\"name\\\":\\\"Some-Name\\\",\\\"domainOrIpAddress\\\":\\\"example.com\\\",\\\"description\\\":\\\"This is a description field\\\",\\\"enabled\\\":\\\"true\\\"}\",\"AuditOperationType\":\"Create\",\"ObjectType\":\"Server\",\"ObjectName\":\"Some-Name\",\"ObjectID\":12345678901234567,\"CustomerID\":98765432109876543,\"User\":\"[email protected]\",\"ClientAuditUpdate\":0}",
"outcome": "success",
"type": [
"creation"
]
Expand Down Expand Up @@ -123,6 +125,72 @@
}
}
}
},
{
"@timestamp": "2025-04-30T16:23:40.000Z",
"client": {
"ip": [
"81.2.69.142",
"81.2.69.144"
]
},
"ecs": {
"version": "8.11.0"
},
"event": {
"category": [
"authentication",
"session"
],
"created": "2025-04-30T16:23:40.000Z",
"id": "12d6eccc-718c-4657-b267-83cc1c3f35f6",
"kind": "event",
"original": "{\"ModifiedTime\":\"\",\"CreationTime\":\"2025-04-30T16:23:40.000Z\",\"ModifiedBy\":288263728720249833,\"RequestID\":\"12d6eccc-718c-4657-b267-83cc1c3f35f6\",\"SessionID\":\"1samau4fwi7xbsf3317mkd5vz\",\"AuditOldValue\":\"\",\"AuditNewValue\":\"{\\\"loginAttempt\\\":\\\"2025-04-30 16:23:40 UTC\\\",\\\"remoteIP\\\":\\\"81.2.69.142, 81.2.69.144\\\"}\",\"AuditOperationType\":\"Sign In\",\"ObjectType\":\"Authentication\",\"ObjectName\":\"xxxx\",\"ObjectID\":\"xxxxx\",\"CustomerID\":\"xxxxx\",\"User\":\"xxxx\",\"ClientAuditUpdate\":1}",
"outcome": "success",
"type": [
"start"
]
},
"organization": {
"id": "xxxxx"
},
"related": {
"ip": [
"81.2.69.142",
"81.2.69.144"
],
"user": [
"288263728720249833",
"xxxx"
]
},
"tags": [
"preserve_original_event"
],
"user": {
"id": "288263728720249833",
"name": "xxxx"
},
"zscaler_zpa": {
"audit": {
"client_audit_update": 1,
"object": {
"id": "xxxxx",
"name": "xxxx",
"type": "Authentication"
},
"operation_type": "Sign In",
"session": {
"id": "1samau4fwi7xbsf3317mkd5vz"
},
"value": {
"new": {
"loginAttempt": "2025-04-30 16:23:40 UTC",
"remoteIP": "81.2.69.142, 81.2.69.144"
}
}
}
}
}
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -39,30 +39,74 @@ processors:
field: event.created
copy_from: '@timestamp'
ignore_failure: true
- append:
field: event.category
value: iam
- set:
field: event.kind
value: event
- script:
if: ctx.json?.AuditOperationType != null && ctx.json.AuditOperationType != ''
lang: painless
params:
event_classification:
'create':
type:
- creation
category:
- iam
outcome: success
'delete':
type:
- deletion
category:
- iam
outcome: success
'update':
type:
- change
category:
- iam
outcome: success
'sign in':
type:
- start
category:
- authentication
- session
outcome: success
'sign in failure':
type:
- start
- error
category: failure
- session
outcome:
'download':
type:
- info
- access
outcome: success
category:
- file
'sign out':
type:
- end
category:
- session
outcome: success
'client session revoked':
type:
- change
- deletion
category:
- iam
outcome: success
source: |
def eventType = ctx.json.AuditOperationType?.toLowerCase();
ctx.event.type = new ArrayList();
Map referenceTable = [
'create': ['creation'],
'delete': ['deletion'],
'update': ['change'],
'sign in': ['access', 'allowed'],
'sign in failure': ['access', 'error'],
'download': ['info'],
'sign out': ['access'],
'client session revoked': ['end']
];

ctx.event.type = referenceTable[eventType];
def class = params.event_classification[ctx.json.AuditOperationType?.toLowerCase()];
if (class == null) {
return;
}
ctx.event.type = class.type;
ctx.event.category = class.category;
ctx.event.outcome = class.outcome;
- rename:
field: json.RequestID
target_field: event.id
Expand Down Expand Up @@ -216,12 +260,25 @@ processors:
ctx.server = new HashMap();
ctx.server.address = valuesMap?.domainOrIpAddress;
}
- split:
field: client.ip
separator: ', *'
if: ctx.client?.ip instanceof String && ctx.client.ip.contains(',')
Comment on lines +263 to +266
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having a list of client.ip/ source.ip will prevent us from using geoip processor on them. Maybe we should find the public IP among the 2 IPs and populate just with it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm following this up to see if there is a known behaviour we can rely on to do this.

- append:
field: related.ip
value: '{{{client.ip}}}'
if: ctx.client?.ip != null
if: ctx.client?.ip instanceof String
allow_duplicates: false
ignore_failure: true
- foreach:
field: client.ip
if: ctx.client?.ip instanceof List
processor:
append:
field: related.ip
value: '{{{_ingest._value}}}'
allow_duplicates: false
ignore_failure: true
- convert:
field: server.address
target_field: server.ip
Expand Down
2 changes: 1 addition & 1 deletion packages/zscaler_zpa/manifest.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
format_version: "3.0.3"
name: zscaler_zpa
title: Zscaler Private Access
version: "1.22.1"
version: "1.22.2"
source:
license: Elastic-2.0
description: Collect logs from Zscaler Private Access (ZPA) with Elastic Agent.
Expand Down