Mirror of forum.nim-lang.org

3343 :: Is there way to just dump staticRead bytes to seq.

• treeform (orginal) [2017-11-16T21:55:05+01:00] view original

  I am working on my timezone library ( https://github.com/treeform/chrono ) and I would like to pack timezone definitions into the binary. I can do that with staticRead, but then I have to read the binary out of the string. My binary uses the exact same flat (no nested structures) as the seq memory. Is there a way for me to just say seq should point to the bytes from staticRead without doing any copies or reads?

  Here is the code I have now which feels super complex (and stupid):

  const zoneData = staticRead("../tzdata/timeZones.bin")
  
  var timeZones* = newSeq[TimeZone](zoneData.len div sizeof(TimeZone)) ## List of all timezones
  
  var zoneStream = newStringStream(zoneData)
  for i in 0..<timeZones.len:
    var dummyZone = TimeZone()
    discard zoneStream.readData(cast[pointer](addr dummyZone), sizeof(TimeZone))
    timeZones[i] = dummyZone
  
  Run

  I want to have code look more like this:

  var timeZones = newSeq[TimeZone](0)
  var timeZonesBin = staticRead("../tzdata/timeZones.bin")
  
  timeZone.internalPointer = addr timeZonesBin[0]
  timeZone.len = timeZonesBin.len div sizeof(TimeZone)
  Run

  Thanks!

---

• cheatfate (orginal) [2017-11-16T22:12:33+01:00] view original

  I'm curious, so if i use your library to handle timezone transitions, my binary will hold +6mb? Also i'm curious why your timezone data is 6mb, while https://momentjs.com/downloads/moment-timezone-with-data.min.js is only 26kb. And even system specific TZ files in my linux distribution occupied only 5Mb.

---

• treeform (orginal) [2017-11-17T00:22:09+01:00] view original

  Its 3MB not 6MB. So its less then linux. And its not compressed. Moment TZ implements a custom compressor for the TZ data. I have though about depending on some thing like snappy to compress the timezones db but that will add ms to startup time and extra dependency.

  Do you think I should add snappy and get it as small as possible?

---

• cheatfate (orginal) [2017-11-17T09:32:58+01:00] view original

  I just want to say that almost every target has its own timezone database, except javascript and standalone. And javascript don't need to have 3mb of data inside of it. Also its hard to maintain, when countries decide to remove DST or change timezone, so maybe it would be better to use system specific databases?

---

• treeform (orginal) [2017-11-17T19:07:05+01:00] view original

  Python (pytz), Java (JDK), Rust (chrono-tz) also maintains its own databases. I don't think its that unusual.

  Can you find an example of a language that uses system timezones on windows through win32 api (not .net)?

---

• cheatfate (orginal) [2017-11-17T23:10:13+01:00] view original

  pytz and chrono-tz not official packages, while Java programs needs JDK (which is already many megabytes installed in your system). So Java don't care about binary size. I dont think you mentioned a good examples how to handle timezones database. As i said we still needs the database for javascript and standalone targets, but people will dislike to have +3Mb binaries in Linux/BSD/Windows/MacOS.

---

• mratsim (orginal) [2017-11-18T00:07:59+01:00] view original

  If gz is enough, can it can be compressed through .gz instead of Snappy? Does Windows require an external decompressor for .gz as well?

---