Skip to content

Computing full effective mappings for data stream mapping APIs #130498

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

masseyke
Copy link
Member

@masseyke masseyke commented Jul 2, 2025

The update data stream mappings API (#130241) validates that the effective mapping after applying the user's mapping overrides is still a valid template. However, the effective mapping is generated using only the mapping overrides and the composable template's top-level mapping. It ignores any mappings provided by component templates within the composable template. This is a bug. This PR fixes that bug, and also uses the same new logic for generating the effective_mappings that are returned to the user in the get and update data stream mappings APIs.

@masseyke masseyke added >non-issue :Data Management/Data streams Data streams and their lifecycles v9.2.0 labels Jul 2, 2025
@masseyke masseyke requested a review from Copilot July 3, 2025 21:32
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds full effective mapping computation for data streams by introducing overloaded mapping methods, updating service and transport layers, and adjusting tests/specs to validate the merged mappings.

  • Introduce new getEffectiveMappings overloads in DataStream requiring IndicesService.
  • Update MetadataDataStreamsService and transport actions to pass an IndicesService into mapping validation.
  • Refresh tests and REST-spec YAML to expect the additional timestamp and merged properties.

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
server/src/test/java/.../DataStreamTests.java Updated test to call new getEffectiveMappings signature
server/src/main/java/.../MetadataDataStreamsService.java Switched validateTemplate call to null mappings (should use new API)
server/src/main/java/.../DataStream.java Added overloaded getEffectiveMappings methods with IndicesService
server/src/main/java/.../ComposableIndexTemplate.java Made merge and convertMappingMapToXContent public
modules/data-streams/src/yamlRestTest/.../250_data_stream_mappings.yml Updated expected property counts and new fields in spec
modules/data-streams/src/main/java/.../TransportUpdateDataStreamMappingsAction.java Injected IndicesService, updated call to getEffectiveMappings
modules/data-streams/src/main/java/.../TransportGetDataStreamMappingsAction.java Injected IndicesService, updated call to getEffectiveMappings
modules/data-streams/src/internalClusterTest/.../TransportUpdateDataStreamMappingsActionIT.java Updated expected mappings to include _data_stream_timestamp and @timestamp
modules/data-streams/src/internalClusterTest/.../DataStreamIT.java Added end-to-end testGetEffectiveMappings integration test
Comments suppressed due to low confidence (1)

server/src/test/java/org/elasticsearch/cluster/metadata/DataStreamTests.java:2692

  • The call to mock(IndicesService.class) requires a static import of org.mockito.Mockito.mock. Please add import static org.mockito.Mockito.mock; to avoid a compilation error.
        IndicesService indicesService = mock(IndicesService.class);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant