Skip to content

ct-fetch: support for tiled logs #359

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jul 21, 2025
Merged

ct-fetch: support for tiled logs #359

merged 11 commits into from
Jul 21, 2025

Conversation

jschanck
Copy link
Collaborator

@jschanck jschanck commented Jul 15, 2025

Resolves #358

@jschanck jschanck changed the title (wip) ct-fetch: support for tiled logs ct-fetch: support for tiled logs Jul 17, 2025
@jschanck jschanck requested a review from mozkeeler July 17, 2025 20:13
Copy link
Collaborator

@mozkeeler mozkeeler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just a couple of questions.

@jschanck jschanck requested a review from mozkeeler July 21, 2025 15:58
@jschanck
Copy link
Collaborator Author

While testing in the dev instance we ran out of memory in Redis (tiled logs are fast...). Turns out there was also a bug where log state updates were sent to Redis on a different thread than certificate entries, so they were not guaranteed to be ordered correctly. Hence, on something like an OOM failure, the log state in redis could be ahead of the log entries that had actually been written to cache.

I've added a couple patches here to fix this. Redis is single-threaded so there's no performance penalty in sequencing operations in insertCTWorker rather than in Redis itself.

@jschanck jschanck merged commit 0b03795 into mozilla:main Jul 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ct-fetch: Add support for tiled logs
2 participants