Iterator support for serde deserialization

[vorner]

I have a file that is a sequence of serialized structs and I want to read them, one by one. I believe this is quite common situation. This is currently a bit hard, I have the following code:

while let Ok(record) = from_read::<_, Record>(&mut reader) {
  // Do stuff with record here
}
// Check that the above loop ended (errored) on eof, not something else
let mut ctl_buf = [0];
assert_eq!(0, reader.read(&mut ctl_buf).unwrap());

This checks first tries to repeatedly read a value from the reader until it fails and then checks that it reached the end of the file, not some other error. This, however, might also mean an EOF in the middle of the record, not only on the boundary (eg. if there's no more record).

There could be two ways to allow handling the situation in a better way:

• Have an error type that can distinguish the situations (eg. an EOF before we read any data, an EOF in the middle of the record…).
• Have an iterator reader, similar to serde-json has in StreamDeserializer.

Would it be hard to add either of these?

[3Hren]

Both solutions requires stateful deserializers, while currently it's stateless. The StreamDeserializer fits better IMO, because it also resolves stack-explosion problem, which is currently resolved by hacking with recursion counter.

Personally I have no near plans for adding another deserializer, because currently I switched for another project, but PR's are welcome!

[vorner]

Thank you for your answer. It is probably by a lack of knowledge of the code that I don't see how having a different error variant when an unexpected EOF happens than in the other cases needs a state, I'll have to read it I guess.

The EOF thing has a different advantage ‒ it can be made to work with asynchronous case (where all the data may or may not yet have arrived).

Anyway, I'll try to get to it some time and provide a PR.

[3Hren]

The EOF thing has a different advantage ‒ it can be made to work with asynchronous case (where all the data may or may not yet have arrived).

Personally I've resolved this by explicit splitting reading and parsing stages. For example: https://github.com/3Hren/cocaine-framework-rust/blob/master/src/lib.rs#L412