Skip to content

amazon_security_lake: fix handling of JSON encoded flattened fields #14323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 30, 2025

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented Jun 26, 2025

Proposed commit message

amazon_security_lake: fix handling of JSON encoded flattened fields

The ocsf.api.{request,response}.data fields may be strings holding JSON
encoded data, but the fields are mapped as flattened, resulting in
failure to index documents. Conditionally JSON decode the string and if
this fails, put the value in a well known location in an object so that
the flattened mapping does not fail.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@efd6 efd6 self-assigned this Jun 26, 2025
@efd6 efd6 added bugfix Pull request that fixes a bug issue Integration:amazon_security_lake Amazon Security Lake Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Jun 26, 2025
@efd6 efd6 force-pushed the s6140-amazon_security_lake-flattened branch from a7a0602 to 274099b Compare June 26, 2025 00:00
The ocsf.api.{request,response}.data fields may be strings holding JSON
encoded data, but the fields is mapped as flattened, resulting in
failure to index documents. Conditionally JSON decode the string and if
this fails, put the value in a well known location in an object so that
the flattened mapping does not fail.
@efd6 efd6 force-pushed the s6140-amazon_security_lake-flattened branch from 274099b to 1105833 Compare June 26, 2025 00:22
@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link

💚 Build Succeeded

cc @efd6

Copy link

@efd6 efd6 marked this pull request as ready for review June 26, 2025 21:38
@efd6 efd6 requested a review from a team as a code owner June 26, 2025 21:38
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

Copy link
Contributor

@ShourieG ShourieG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@efd6 efd6 merged commit 8e95cee into elastic:main Jun 30, 2025
7 checks passed
@elastic-vault-github-plugin-prod

Package amazon_security_lake - 2.5.2 containing this change is available at https://epr.elastic.co/package/amazon_security_lake/2.5.2/

shmsr pushed a commit to shmsr/integrations that referenced this pull request Jun 30, 2025
…lastic#14323)

The ocsf.api.{request,response}.data fields may be strings holding JSON
encoded data, but the fields is mapped as flattened, resulting in
failure to index documents. Conditionally JSON decode the string and if
this fails, put the value in a well known location in an object so that
the flattened mapping does not fail.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix Pull request that fixes a bug issue Integration:amazon_security_lake Amazon Security Lake Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants