nimforum mirror - GuildenStern web server 6.0.0 is here

Allin (orginal) [2023-08-23T18:14:24+02:00] view original

So far GuildenStern has been a bit of an experimental project, but with Nim 2 and this major rewrite, I hope I have nailed it, and the API stays more stable for the future generations to come. Please try it out and send bug reports. After those ease off, I am inclined to call this production-ready.

https://github.com/olliNiinivaara/GuildenStern

termer (orginal) [2023-08-23T18:31:05+02:00] view original

I'd love to stream building an application with this like I did for Mummy. Interested to see how it compares, especially on performance.

ITwrx (orginal) [2023-08-23T21:49:34+02:00] view original

i see you have a streaming server example. does this mean a large file upload will get saved to disk without using ram equivalent to file size? If so, can i serve large files with the same benefit?

Allin (orginal) [2023-08-24T09:09:49+02:00] view original

yes, upload from client to server works with StreamingServer. For downloading big data from server to client, there are at least 5 options, each with their own pros and cons:

The replyStart-replyMore-replyFinish -combo supports sending response in parts. But it is naive in the sense that replyStart requires contentlength as parameter.

Http upload is symmetric to download, right? In that case it should be easy to write replyStartMultipart proc which sets appropriate headers and replyFinishMultipart proc which writes the boundary delimiter to the end. If this is what you need, open an issue.

Delivery of all static files should be done by your reverse proxy, which offers caching, compression, HTTP/3 and other sophisticated features that are not in the scope of an application-side web server.

Even if data is dynamic and/or private, it could be handled by reverse proxy using something like this: 1. when request for data arrives, write it to disk, optionally compress it, and give it a large random filename that is impossible to guess. Send this filename back to client. 2. client asks the file from reverse proxy which serves it as any static file. 3. after transfer is ready, client signals this (or after a timeout) and you can remove the file.

Send the data using websocket stream.

Hope this helps.

ITwrx (orginal) [2023-08-24T17:38:47+02:00] view original

Thanks for the detailed response.

It looks like replyStart-replyMore-replyFinish -combo uses the http server as opposed to the streaming server, so i'm guessing it would use a bunch of ram for a larger video file for instance?

i don't fully understand this one. Downloads would not be happening right after upload if that is the assumption.

I use nginx for normal static resources, but some files have different needs.

i already use generated urls, rate limiting to prevent guessing, etc. for some private downloads in some apps, but i want to write a personal cloud software (as one example), so it would be lots of files available through more complex logic than normal, and i don't want a complicated, brittle, not-so-secure access system between two servers, nor strange file names visible to the user for their own files. I'm sure some system could be devised, but with what i've seen and used so far, i'm thinking it would be better to just use the app server for this specific use case, however the serving would need to work under the hood.

I'll look into that too, but i suspect it's not ideal for serving large files.

Multi-threaded servers address the crashy-ness of my previous attempts with nim async servers/frameworks, but low ram file upload and serving has been missing. It looks like Guildenstern addresses the upload so far. I would like to use one base for all my web projects, and am not low level enough to write my own http+websocket server or contribute this feature to an existing one at this time. I would rather focus on my apps' business logic, really. :)

Allin (orginal) [2023-08-25T13:21:54+02:00] view original

The documentation is lacking here. StreamingContext is a subclass of HttpContext, so every way you can reply with httpserver, you can reply with streamingserver. So this should already solve your use case.

Allow me to replace the term "symmetric" with term "identical". I hope this rephrasing makes the sentence more sensible.

Well no one can foretell all special file needs you have, and likely quite few are keen to code them for you (for free). If you are implementing some non-standard service, you have to be prepared to get your hands dirty and write some code yourself.

At least I can say that if you use 64 char set and 200 char long file names, files are protected by 1200 bit key, which is strong enough for you. Also hiding the real names used during file transfer should be a case easily handled by UI. Everything else is more complicated than letting reverse proxy serve the static files.

5. Now that you reveal a bit more info about your intended use case, it comes to mind that there are also protocols that are especially designed for sharing large files. Check at least: https://github.com/status-im/nim-libp2p and https://github.com/sergiotapia/torrentinim

Anyway, GuildenStern is a good platform for developing customized services that are optimized for particular use cases, thanks to its modularity that allows easy replacement of components, and architecture, that allows different kinds of servers to co-exist serving different ports. GuildenStern is not (just) a web server, it is a web server construction kit. With it, you will never paint yourself into the corner.

ITwrx (orginal) [2023-08-25T18:19:48+02:00] view original

great! Sounds good.

Just to be clear: i'm not expecting anyone to code anything. Just trying to find out what Guildenstern can do in relation to my use case.

Thanks for all the info.

Mirror of forum.nim-lang.org

10433 :: GuildenStern web server 6.0.0 is here