I'm new to okio and haven't used it in anger yet, but something I'm building now can likely reuse Buffer of okio.
I'm working with a serialization protocol/codec that works directly on a buffer, essentially i will receive messages which are normally structs that i can literally wrap a segment of a buffer and treat it as the message i work with. Its a flyweight pattern which basically allows me to read from network card to buffer, directly manipulate data on the buffer and send it without creating bunch of objects and actually being faster than anything else i know out there. No time spent on actual serialization or deserialization.
There are few challenges here :
-
reuse of the underlying byte[]/ByteBuffer i use, and their variable size per message, while most of the messages are small lets say less than 128 bytes, there may in theory be messages that are megabytes in size (snapshots of some kind). Obviously i do not want to maintain a pool of byte[] with the maximum size observed...
-
as i "deserialize/wrap" a message i may randomly read/write at different offsets but always within some specific bounds, most messages are structs so i know exactly where to read/write and how many bytes.
I know that okio Buffer is something that works with segments under the hood but i couldn't see a good API to use them.
Ideally i should be able to do something like buffer.allocateSegment(numberOfBytes)... get a segment object (which ideally is not thread safe) so all interactions i do with the buffer are thread confined... once I'm done with the segment i should be able to return it and let the buffer mange memory...
Are there any examples that i could use to see how to achieve this with okio Buffer? There may be multiple segments that are allocated and actively used at a given time.
P.S. is there a way to treat okio Buffer as a standard byte buffer? it does not seem to extend/implement anything that could be used that way.
I understand that java devs treat us like children and seal ByteBuffer so now i have to go trough trouble if i want to make something that works with ByteBuffer to work with Buffer... its the same annoying story if i want to use chronicles Bytes or Aerons buffer equivalents...
How do you deal with libraries that rely on nio buffers?
I'm new to okio and haven't used it in anger yet, but something I'm building now can likely reuse Buffer of okio.
I'm working with a serialization protocol/codec that works directly on a buffer, essentially i will receive messages which are normally structs that i can literally wrap a segment of a buffer and treat it as the message i work with. Its a flyweight pattern which basically allows me to read from network card to buffer, directly manipulate data on the buffer and send it without creating bunch of objects and actually being faster than anything else i know out there. No time spent on actual serialization or deserialization.
There are few challenges here :
reuse of the underlying byte[]/ByteBuffer i use, and their variable size per message, while most of the messages are small lets say less than 128 bytes, there may in theory be messages that are megabytes in size (snapshots of some kind). Obviously i do not want to maintain a pool of byte[] with the maximum size observed...
as i "deserialize/wrap" a message i may randomly read/write at different offsets but always within some specific bounds, most messages are structs so i know exactly where to read/write and how many bytes.
I know that okio Buffer is something that works with segments under the hood but i couldn't see a good API to use them.
Ideally i should be able to do something like buffer.allocateSegment(numberOfBytes)... get a segment object (which ideally is not thread safe) so all interactions i do with the buffer are thread confined... once I'm done with the segment i should be able to return it and let the buffer mange memory...
Are there any examples that i could use to see how to achieve this with okio Buffer? There may be multiple segments that are allocated and actively used at a given time.
P.S. is there a way to treat okio Buffer as a standard byte buffer? it does not seem to extend/implement anything that could be used that way.
I understand that java devs treat us like children and seal ByteBuffer so now i have to go trough trouble if i want to make something that works with ByteBuffer to work with Buffer... its the same annoying story if i want to use chronicles Bytes or Aerons buffer equivalents...
How do you deal with libraries that rely on nio buffers?