-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use parquet metadata cache for parquetmetadata format as well #636
base: antalya
Are you sure you want to change the base?
use parquet metadata cache for parquetmetadata format as well #636
Conversation
namespace DB | ||
{ | ||
|
||
ParquetFileMetaDataCache::ParquetFileMetaDataCache() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks too silly
@@ -129,10 +143,35 @@ void checkHeader(const Block & header) | |||
static std::shared_ptr<parquet::FileMetaData> getFileMetadata( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are two implementations of getFileMetadata
. One for ParquetBlockInputFormat.cpp
and one for ParquetMetadataInputFormat
. I am still not sure if I should leave it duplicated or put it somewhere.. The logic is pretty much the same, but requires a couple of arguments to be passed
175a172
to
cb8d032
Compare
This is an automated comment for commit 79da2cf with description of existing statuses. It's updated for the latest CI running ❌ Click here to open a full report in a separate page
Successful checks
|
Previous implementation #586 only applied caching to blockinput format. In order to re-use and share the cache, some refactoring had to be done. Cache initialization was moved to
Server.cpp
.Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Use ParquetMetadataCache for ParquetMetadata format as well.
Documentation entry for user-facing changes
Modify your CI run:
NOTE: If your merge the PR with modified CI you MUST KNOW what you are doing
NOTE: Checked options will be applied if set before CI RunConfig/PrepareRunConfig step
Include tests (required builds will be added automatically):
Exclude tests:
Extra options:
Only specified batches in multi-batch jobs: