Mirror of forum.nim-lang.org

808 :: Manually managing a block of memory

• andrea (orginal) [2015-01-30T10:41:42+01:00] view original

  I have a fairly large block of memory (the content of a big file). What I would like to do is to split this block into n smaller blocks, where n is the number of cores, and perform a certain operation in parallel on each block.

  First, I will need to read the whole thing into memory. I can either:

  • mmap the file, createU a large enough block, and use copyMem to copy the memory (mmaping itself won't be enough, since I will have to perform destructive operations on the memory block)
  • use readFile to read the whole file

  The second seems simpler, but it return a string. Is this allocated consecutively, so that I obtain the same result? Does it require more memory (the first strategy does not require 2x the memory, since the OS can page out the mmapped memory after it's been copied)?

  The first strategy, instead, returns a size and a raw pointer. Is there a way to read past this pointer? I have tried p[10] but this does not compile

---

• fadg44a3w4fe (orginal) [2015-01-31T03:41:54+01:00] view original

  re mmap: Can't you use the MAP_PRIVATE flag to force the OS to worry about copying? "Updates to the mapping are not visible to other processes mapping the same file, and are not carried through to the underlying file."

  re indexing: You can use http://nim-lang.org/manual.html#unchecked-pragma to do the indexing you describe, but you'll have to manage your own bound checking.

---