I found this code for uploading files, but it only works well for small files. With large files (100MB) it’s very slow and my cpu goes up to close to 100%!
import os, re, jester, asyncdispatch, htmlgen, asyncnet
routes:
get "/":
var html = ""
for file in walkFiles("*.*"):
html.add "<li>" & file & "</li>"
html.add "<form action=\"upload\" method=\"post\"enctype=\"multipart/form-data\">"
html.add "<input type=\"file\" name=\"file\"value=\"file\">"
html.add "<input type=\"submit\" value=\"Submit\" name=\"submit\">"
html.add "</form>"
resp(html)
post "/upload":
writeFile("uploaded.png", request.formData.getOrDefault("file").body)
resp(request.formData.getOrDefault("file").body)
runForever()
It seems the problem is elsewhere, not on how writing it to file, whether sync/async file write.
Idk what to tweak on Jester to support larger chunk processing. As for example on writeFromStream (not actually solving the problem tho):
routes:
post "/upload":
var f: AsyncFile
var fstream = newFutureStream[string]("routes.upload")
try:
f = openAsync("uploaded.file", fmWrite)
except IOError:
echo getCurrentExceptionMsg()
resp Http500, "Cannot upload file"
return
defer: f.close
var datastream = newStringStream(request.formData.getOrDefault("file").body)
var asyncwrite = f.writeFromStream(fstream)
while not datastream.atEnd:
# read each of 1 MB
let strdata = datastream.readStr(1024 * 1024)
await fstream.write strdata
fstream.complete
resp Http200, "uploaded"
Uploading file has been tricky, but thankfully nowadays there's websocket to send blob. Here's example how to send a file.
import os, jester, asyncdispatch, htmlgen, asyncfile, asyncstreams, streams
import strutils
import ws, ws/jester_extra
settings:
port = Port 3000
routes:
get "/":
var html = """
<script>
function submit_file() {
let ws = new WebSocket("ws://localhost:3000/ws-upload");
let filedom = document.querySelector("#input-field");
ws.onmessage = function(evnt) {
console.log(evnt.data);
}
ws.onopen = function(evnt) {
ws.send(filedom.files[0].name);
ws.send(filedom.files[0].slice());
ws.close();
}
return true;
}
</script>
"""
for file in walkFiles("*.*"):
html.add "<li>" & file & "</li>"
html.add "<form action=\"upload\" method=\"post\"enctype=\"multipart/form-data\">"
html.add "<input id=\"input-field\" type=\"file\" name=\"file\" value=\"file\">"
html.add "<input type=\"button\" value=\"Submit\" name=\"submit-button\" onclick=\"submit_file()\">"
html.add "</form>"
resp(html)
get "/ws-upload":
try:
var wsconn = await newWebSocket(request)
await wsconn.send("send the filename")
var fname = await wsconn.receiveStrPacket()
var f = openAsync(fname, fmWrite)
while wsconn.readyState == Open:
let (op, seqbyte) = await wsconn.receivePacket()
if op != Binary:
resp Http400, "invalid sent format"
wsconn.close()
return
var cnt = 0
if seqbyte.len < 4096:
await f.write seqbyte.join
continue
while cnt < (seqbyte.len-4096):
let datastr = seqbyte[cnt .. cnt+4095].join
cnt.inc 4096
await f.write(datastr)
wsconn.close()
f.close()
except:
echo "websocket close: ", getCurrentExceptionMsg()
resp Http200, "file uploaded"
it seems there is a offset error in the code (files are not identically but shifted by some bytes at certain offsets)
This is quite tricky to find out, TBH. I tried running it on windows and debian (in virtual box) and using chromium and firefox (both on windows) and it gave the correct files. I hadn't tried with linux box yet though.
Will the websocket work if I use the nginx as a frontend with ssl?
it's supposed to be working fine, nginx would handle the handshake and the apps would only receive the socket connection, I remembered there's some option to be set to enable websocket on nginx, but never actually tried it too tho.