Tap/raw recorder#1
Open
krowd3v wants to merge 4 commits into
Open
Conversation
Adds opt-in byte-level recording to the generic TCP forwarder. A recorder
configured with `record: recorder.service.handler.raw` (new constant) is
wired through a recorderConn wrapper — same pattern as handler/serial —
that emits one record per Read/Write with the standard Direction /
Hexdump / TimestampFormat options from the core recorder.Options struct.
Use case: plaintext protocols that GOST's sniffer doesn't recognize
(WHOIS, DNS-over-TCP, port-43 registry protocols) can now be captured
through the existing recorder pipeline without a protocol-specific
handler fork and without client-side instrumentation.
Coexists with the default recorder.service.handler (one JSON per
request with route/duration/bytes). A service can attach both: raw
for payload, handler for metadata.
Files:
- recorder/recorder.go +1 const RecorderServiceHandlerRaw
- handler/forward/local/conn.go +new recorderConn (mirror of
handler/serial/conn.go pattern)
- handler/forward/local/handler.go
- new rawRecorder field on forwardHandler
- Init picks up the new record type
- Handle wraps the incoming conn when rawRecorder is set
Three small enhancements on top of the raw-bytes recorder patch.
listener/grpc:
- New metadata key `grpc.maxStreams` / `maxStreams` (uint). When set,
passes `grpc.MaxConcurrentStreams(n)` to the embedded grpc.NewServer.
grpc-go default is math.MaxUint32 (unlimited) but the server does
not advertise that in SETTINGS, so clients brief-cap at 100 streams
until the first settings frame arrives (http2_client.go:349). Setting
this explicitly makes the server advertise, eliminating the transient
cap for the first ~RTT after a ClientConn is established.
- Unix domain socket support. `addr: unix:///tmp/foo.sock` or `addr:
/path/to.sock` / `addr: ./path.sock` switches the listener's network
to "unix" instead of "tcp". Stale socket files are removed before
bind, matching other GOST unix-capable listeners. Saves ~20% syscall
overhead vs loopback TCP at high message rates.
handler/forward/local (raw recorder):
- Embed the session id (SID, same one used in the meta
recorder.service.handler JSON) in every raw record emitted by the
wrapped recorderConn. Prefix format:
sid:<sid>\n
<direction>[timestamp]\n (optional, per Options)
<payload> (raw bytes or hex.Dump)
The SID lets the receiver join raw-byte records with the parent
metadata record server-side without relying on timestamp-window or
proxy-node heuristics. Backward compatible: when SID is empty the
prefix line is omitted.
krowd3v
force-pushed
the
tap/raw-recorder
branch
2 times, most recently
from
a82c001 to
113f4a0
Compare
Extends the raw-recorder preamble so every record self-describes: a
receiver can persist a complete row keyed by sid without cross-referring
to the sibling meta record.
Record format (all context lines optional; appear only when known):
sid:<sid>\n session id (matches meta record sid)
route:<route>\n proxy-<hash>@<ip>:<port> > <target>
(same string as meta record route)
svc:<name>\n GOST service name
<direction>[timestamp]\n direction + optional fmt timestamp
<payload> raw bytes (or hex.Dump)
route is the most useful addition: it identifies which proxy node the
chain selected for THIS request, inline, so a sink can attribute bytes
to a proxy without joining a meta record first.
The route is pushed into the wrapper by the handler after Router.Dial
resolves the chain (ro.Route is already computed there for the meta
record -- same string, zero extra work).
Backward-compatible: fields without a known value are simply omitted;
no new metadata knobs to configure.
Owner
Author
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c560cf1537
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
62
to
+66
| for _, ro := range h.options.Recorders { | ||
| if ro.Record == xrecorder.RecorderServiceHandler { | ||
| switch ro.Record { | ||
| case xrecorder.RecorderServiceHandler: | ||
| h.recorder = ro | ||
| break | ||
| case xrecorder.RecorderServiceHandlerRaw: |
There was a problem hiding this comment.
This loop no longer stops after finding recorder.service.handler, so the last matching entry now overwrites earlier ones. Other handlers in this repo keep first-match behavior (break on first match), so configs with multiple handler recorder entries will silently change which sink receives service metadata after this commit. That regression is hard to detect and can redirect records to the wrong backend.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.