Mirror of forum.nim-lang.org

2551 :: Stream data type

• andrea (orginal) [2016-09-23T20:12:01+02:00] view original

  In Rosencrantz I am trying to implement support for streaming responses with transfer econding chunked.

  These can be used to stream large files, or to represent the case where the data for the response comes piecewise.

  The handler to produce these responses should get as input some kind of stream. What datatype should I use to represent the stream? If all data was available at once, I could use an iterator. But I see that the support for async operations is also implemented on top of iterators, which are used to simulate coroutines.

  I have the feeling that an iterator can somehow be used to represent an asynchronous stream - by making some smart ping pong between it and the procedure that consumes it - but I cannot quite put it down clearly.

  Any ideas?

---

• zielmicha (orginal) [2016-09-23T22:36:43+02:00] view original

  You might want to look at https://github.com/zielmicha/reactor.nim/blob/master/reactor/async/stream.nim from my reactor.nim - it implements asynchronous streams. The implementation is a bit involved (mostly for performance and to properly implement stream closing), but in it's core it just invokes receive callbacks when data is written to it. You could implement something really similar on top of async API from stdlib.

---

• pdv (orginal) [2016-09-26T05:08:37+02:00] view original

  There is a new asyncstreams module in nimboost. It provides API close to streams module in stdlib, so it's probably too low-level for your standards, though.

  As for a cleaner abstraction, you might want to look into iteratees for that - seems close to what you're talking about.

---