Skip to content

azure: add related.entity field to activitylogs default ingest pipeline #11233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Nov 6, 2024

Conversation

orouz
Copy link
Contributor

@orouz orouz commented Sep 24, 2024

Proposed commit message

adds processors to activitylogs ingest pipeline that appends principal_id and resource_id to related.entity in order to facilitate pivoting around a piece of data

Context

this PR is part of the cloud security CDR epic. it populates related.entity which is an upcoming ECS field

Related issues


benchmark (30 runs)

Metric      
Description Main append processor painless processor
Commit 00ed51b 21b7189 bd6cde2
Average EPS 10333.47787 9546.633648 10726.24097
Stddev 1549.043495 1967.80285 1281.125397
Min 3460.207612 3759.398496 6211.180124
Max 11764.70588 11627.90698 11627.90698
EPS Change to baseline   -7.61% 3.80% (?)

@orouz orouz added enhancement New feature or request Integration:azure Azure Logs labels Sep 24, 2024
@orouz orouz force-pushed the azure_activitylogs_cdr_pipeline branch from 47f2eea to 21b7189 Compare September 24, 2024 12:33
@orouz
Copy link
Contributor Author

orouz commented Sep 25, 2024

/test

@orouz orouz force-pushed the azure_activitylogs_cdr_pipeline branch 2 times, most recently from f34cd4a to da652ec Compare September 26, 2024 15:18
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Sep 26, 2024

🚀 Benchmarks report

Package azure 👍(7) 💚(2) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
auditlogs 2409.64 1908.4 -501.24 (-20.8%) 💔
graphactivitylogs 1510.57 1209.19 -301.38 (-19.95%) 💔

To see the full report comment with /test benchmark fullreport

@orouz orouz marked this pull request as ready for review September 26, 2024 15:42
@orouz orouz requested review from a team as code owners September 26, 2024 15:42
@andrewkroh andrewkroh added the Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] label Sep 27, 2024
@andrewkroh andrewkroh requested a review from a team October 7, 2024 13:54
@efd6 efd6 changed the title Add related.entity field to azure activitylogs default ingest pipeline azure: add related.entity field to activitylogs default ingest pipeline Oct 10, 2024
@orouz
Copy link
Contributor Author

orouz commented Oct 20, 2024

/test benchmark fullreport

@efd6
Copy link
Contributor

efd6 commented Oct 20, 2024

@orouz Please revise the proposed commit message to be a stand-alone text that explains what is being done and why. It should preferably not refer to internal issues since it will be read by people who may not have access to those documents. Also note that git commit messages are not markdown, so do not use markdown features.

Copy link
Contributor

@muthu-mps muthu-mps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please resolve the review comments.
Change looks good to me!

@orouz orouz force-pushed the azure_activitylogs_cdr_pipeline branch from 4776367 to 1f35fa8 Compare November 6, 2024 10:42
@orouz orouz requested a review from kaiyan-sheng November 6, 2024 16:09
@elasticmachine
Copy link

💚 Build Succeeded

History

Copy link

Quality Gate failed Quality Gate failed

Failed conditions
57.1% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

@orouz orouz merged commit e368bfe into elastic:main Nov 6, 2024
4 of 5 checks passed
@elastic-vault-github-plugin-prod

Package azure - 1.19.0 containing this change is available at https://epr.elastic.co/search?package=azure

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Integration:azure Azure Logs Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants