Skip to content

add parquet labels cache #6835

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

yeya24
Copy link
Contributor

@yeya24 yeya24 commented Jun 23, 2025

What this PR does:

Add Parquet Labels file cache using the caching bucket. This allows caching Parquet Labels File GetRange requests to reduce API calls to the object storage.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@yeya24
Copy link
Contributor Author

yeya24 commented Jun 23, 2025

--- FAIL: TestBlocksCleaner_ShouldRemoveBlocksOutsideRetentionPeriod (0.16s)
    blocks_cleaner_test.go:700: 
        	Error Trace:	/__w/cortex/cortex/pkg/compactor/blocks_cleaner_test.go:700
        	            				/__w/cortex/cortex/pkg/compactor/blocks_cleaner_test.go:825
        	Error:      	Not equal: 
        	            	expected: false
        	            	actual  : true
        	Test:       	TestBlocksCleaner_ShouldRemoveBlocksOutsideRetentionPeriod
    blocks_cleaner_test.go:830: 
        	Error Trace:	/__w/cortex/cortex/pkg/compactor/blocks_cleaner_test.go:830
        	Error:      	Received unexpected error:
        	            	 # HELP cortex_bucket_blocks_count Total number of blocks in the bucket. Includes blocks marked for deletion, but not partial blocks.
        	            	 # TYPE cortex_bucket_blocks_count gauge
        	            	-cortex_bucket_blocks_count{user="user-1"} 2
        	            	+cortex_bucket_blocks_count{user="user-1"} 1
        	            	 cortex_bucket_blocks_count{user="user-2"} 2
        	            	 # HELP cortex_bucket_blocks_marked_for_deletion_count Total number of blocks marked for deletion in the bucket.
        	            	 # TYPE cortex_bucket_blocks_marked_for_deletion_count gauge
        	            	 cortex_bucket_blocks_marked_for_deletion_count{user="user-1"} 0
        	            	 cortex_bucket_blocks_marked_for_deletion_count{user="user-2"} 0
        	            	 # HELP cortex_compactor_blocks_marked_for_deletion_total Total number of blocks marked for deletion in compactor.
        	            	 # TYPE cortex_compactor_blocks_marked_for_deletion_total counter
        	            	 cortex_compactor_blocks_marked_for_deletion_total{reason="retention",user="user-1"} 1
        	            	 cortex_compactor_blocks_marked_for_deletion_total{reason="retention",user="user-2"} 0
        	            	 
        	Test:       	TestBlocksCleaner_ShouldRemoveBlocksOutsideRetentionPeriod
    blocks_cleaner_test.go:700: 
        	Error Trace:	/__w/cortex/cortex/pkg/compactor/blocks_cleaner_test.go:700
        	            				/__w/cortex/cortex/pkg/compactor/blocks_cleaner_test.go:862
        	Error:      	Not equal: 
        	            	expected: false
        	            	actual  : true
        	Test:       	TestBlocksCleaner_ShouldRemoveBlocksOutsideRetentionPeriod
    blocks_cleaner_test.go:867: 
        	Error Trace:	/__w/cortex/cortex/pkg/compactor/blocks_cleaner_test.go:867
        	Error:      	Received unexpected error:
        	            	 # HELP cortex_bucket_blocks_count Total number of blocks in the bucket. Includes blocks marked for deletion, but not partial blocks.
        	            	 # TYPE cortex_bucket_blocks_count gauge
        	            	-cortex_bucket_blocks_count{user="user-1"} 2
        	            	+cortex_bucket_blocks_count{user="user-1"} 1
        	            	 cortex_bucket_blocks_count{user="user-2"} 0
        	            	 # HELP cortex_bucket_blocks_marked_for_deletion_count Total number of blocks marked for deletion in the bucket.
        	            	 # TYPE cortex_bucket_blocks_marked_for_deletion_count gauge
        	            	 cortex_bucket_blocks_marked_for_deletion_count{user="user-1"} 0
        	            	 cortex_bucket_blocks_marked_for_deletion_count{user="user-2"} 0
        	            	 # HELP cortex_compactor_blocks_marked_for_deletion_total Total number of blocks marked for deletion in compactor.
        	            	 # TYPE cortex_compactor_blocks_marked_for_deletion_total counter
        	            	 cortex_compactor_blocks_marked_for_deletion_total{reason="retention",user="user-1"} 1
        	            	 cortex_compactor_blocks_marked_for_deletion_total{reason="retention",user="user-2"} 2
        	            	 
        	Test:       	TestBlocksCleaner_ShouldRemoveBlocksOutsideRetentionPeriod
FAIL
FAIL	github.com/cortexproject/cortex/pkg/compactor	78.746s

Flaky test worth taking a closer look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant