Skip to content

Conversation

@planthaber
Copy link
Member

test branch to test improvement to reading network-mounted log files

@planthaber planthaber marked this pull request as draft October 24, 2025 11:53
FileStream fileStream;
//FileStream fileStream;
std::ifstream fileStream;
char ReadBuffer[16777216]; //16MB
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would avoid having objects that large (potentially) on the stack.
Also, did you also try making blocksize of pocolog_cpp::FileStream configurable?

@chhtz
Copy link
Member

chhtz commented Oct 27, 2025

If the access pattern is very random, it might also be interesting to try mmap instead of reading blocks of the file manually:
https://stackoverflow.com/questions/45972/mmap-vs-reading-blocks
(No idea how this performs for large files over networks, how well it prefetches memory, etc)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants