Hey all,
I'm primarily a JavaScript developer, fairly new to Nim but really enjoy using it. I'm playing around with some test code to learn the language a little better and right now am specifically playing with staticRead and Zip files.
In my code I staticRead a zip file:
const base = staticRead("./static.zip")
I know that I can write the file and then begin to play with the contents with:
writeFile("./file.zip", base)
But is there a way to use the zip/zipfiles library to examine the contents straight from the staticRead? Something like:
const base = staticRead("./static.zip")
var z: ZipArchive
if open(z, base, fmRead):
try:
for file in walkFiles(z):
echo file
finally:
close(z)
Or is my best option to write the static.zip to temp and then cleanup when done with it?
The zip library relies on a C library (gzip) and C libraries are not available available at compile-time.
Wouldn't runtime be enough (with a let instead of a const)?
Now if you really feel very adventurous, you can try the non-documented compile-time FFI with -d:nimHasLibFFI --experimental:compiletimeFFI but here be dragons. Only one person is using it, so if there is a bug you will be on your own.
I looked at the source: https://github.com/nim-lang/zip/blob/master/zip/zipfiles.nim
And there appears to be no way to use the zipfile interface on a memory buffer. The zlib c library the system wraps seem to use the low level c file stuff so it needs a real file.
Why do you need to static read?
const base = staticRead("./static.zip")
basically turns into a big blob of binary data:
const base = "\x0F\xAD\xBD\x32..."
If the zip file gets big the compile times get long.
I would ship the zipfile along with your excitable not inside it. Many games do that for example they have .exe and some sort of .dat file. You have to ship the zlib.dll and all other dlls with your program anyways... whats one extra file?
staticRead is great for small files (<10k) but I think its just not workable for zipfiles.
@mratsim Appreciate the reply. I started this test after I was asked by an Exec at our company: "Can you take the database, and dump it into an Excel Spreadsheet". We fulfilled his request using many already existing tools but it got me going down the rabbit hole of what exactly are .xlsx files and how could I "generate my own".
After discovering they are essentially .zip files containing a bunch of .xml files, I began playing with the idea of solving his request with Nim (from scratch - for educational purposes). I learned how to create a bare-minimum .xlsx file (5 files required), and discovered that you can bundle files into the Nim binary. My thoughts were to bundle the ~6kb worth of files in an uncompressed Zip, then later pull them out add the data needed and produce a freshly updated .xlsx file on demand - all from a single Nim binary.
I know I could just take the single binary and write a temp zip, edit, produce a .xlsx then clean-up the zip. It just got me wondering if I needed to do all of those steps.
@treeform I was poking through the source myself and that was my hunch - unable to use the zipfiles lib with the memory buffer. I'm not building any products right now, just picking a few "features" from the docs to experiment with and given recent work requests, put together a little mock project to explore.
Normally I wouldn't bother, but I thought the ZIP only being ~6kb in size made it something worth investigating.
Thanks for the help, on to the next project!
It is a pretty common and convenient pattern in my code to "slurp" (an alias for staticRead()) in resource files and write them out only when I need to use them. Most commonly, I pull in SQL scripts for configuration that I may use only one time setup on initial execution. Other resource or default configuration files get written out only the first time. I even manage cross-platform dependencies when I pull in binary files - for example, cross compiling for ARM from desktop is painless by doing something like this that automatically pulls in the correct binary dependency for the target platform. hostCPU() and hostOS() are both convenient standard nim lib functions.
const respath* = "../../res" / hostCPU / hostOS
const myres = respath / "mybinarydependency"
const mybin = slurp myres
...
echo execShellCmd(mybin)