Skip to content

crowdstrike: improve falcon data stream document collision behaviour #13779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

crowdstrike: improve falcon data stream document collision behaviour #13779

wants to merge 1 commit into from
+9 −1

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented May 5, 2025 •
edited
Loading

Proposed commit message

crowdstrike: improve falcon data stream document collision behaviour

The current set of fields that is used to define the _id for documents
is too small and the values are too dense to provide reasonable
guarantees that different documents will be assigned different IDs, so
increase the set to include fields that in conjunction should give good
de-collision behaviour:

* crowdstrike.metadata.offset
* crowdstrike.event.PID
* crowdstrike.event.RuleId

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@efd6 efd6 added enhancement New feature or request Integration:crowdstrike CrowdStrike Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels May 5, 2025
@efd6 efd6 self-assigned this May 5, 2025
@efd6 efd6 force-pushed the 13720-crowdstrike branch from de5957b to 145e72c Compare May 5, 2025 06:04
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented May 5, 2025 •
edited
Loading

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@efd6 efd6 marked this pull request as ready for review May 6, 2025 00:09
@efd6 efd6 requested a review from a team as a code owner May 6, 2025 00:09
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@efd6 efd6 force-pushed the 13720-crowdstrike branch from 145e72c to 61adc64 Compare May 6, 2025 06:16
@efd6
crowdstrike: improve falcon data stream document collision behaviour
a249bb9 
The current set of fields that is used to define the _id for documents
is too small and the values are too dense to provide reasonable
guarantees that different documents will be assigned different IDs, so
increase the set to include fields that in conjunction should give good
de-collision behaviour:

* crowdstrike.metadata.offset
* crowdstrike.event.PID
* crowdstrike.event.RuleId
@efd6 efd6 force-pushed the 13720-crowdstrike branch from 61adc64 to a249bb9 Compare May 6, 2025 10:30
@elasticmachine
Copy link

💚 Build Succeeded

History

cc @efd6

@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@efd6 efd6

Labels
enhancement New feature or request Integration:crowdstrike CrowdStrike Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CrowdStrike]: Processing of different events can lead to identical _ids
2 participants
@efd6 @elasticmachine