Mirror of forum.nim-lang.org

11091 :: Can ref variables reference non-ref variables?

• DMisener (orginal) [2024-02-26T15:25:00+01:00] view original

  The following example almost works but I seem to be missing the magic incantation to get a safe reference to a non-ref content.

  The following examples illustrates my conundrum:

  # I am trying to generate record references of already existing objects in sequence
  # (as to not duplicate the objects).  I am having difficulty figuring out how to set
  # store reference to non-ref items (Things) in a seq of references (RefThings).
  
  import algorithm, std/typeinfo
  
  const WORKING = false
  
  type
    Thing = object
      name: string
      word: string
      originalAddr: string
    
    RefThing = ref Thing
  
  var thingRefs: seq[RefThing]
  
  template memAddr(thing: untyped): string = $cast[uint](thing)
  
  proc showRefThings(title: string) =
    echo "\n", title
    for thingRef in thingRefs.mitems:
      echo memAddr thingRef, ": ", thingRef[]
  
  template showStruct(thing: untyped) =
    let name = thing.name
    echo "\n", name, " structure:"
    echo "Address: ", thing.memAddr
    echo "kind:    ", thing.toAny.kind
    echo "size:    ", thing.toAny.size
    echo "content: ", thing[]
  
  template makeRef(dest: untyped, source: untyped) =
    var dest: RefThing; new dest
    #***********************************
    when WORKING:
      dest[] = source # Actually allocates a new copy? :-(
    when not WORKING:
      dest = cast[RefThing](addr source)
    #***********************************
    dest.originalAddr = source.addr.memAddr
    showStruct dest
  
  
  # *** Minimal failing test case ***
  
  var # Setup Non-ref originals
    original1 = Thing(name: "Thing1", word: "CCC")
    original2 = Thing(name: "Thing2", word: "BB")
    original3 = Thing(name: "Thing3", word: "A")
  
  # Setup references to original items
  makeRef thing1, original1
  makeRef thing2, original2
  makeRef thing3, original3
  
  var things = [thing1, thing2, thing3] # references
  
  # Store reversed refs in the the sequence
  for thing in things.reversed: thingRefs.add thing
  
  # Simple payload on sequece of refs
  showRefThings "Reversed:"
  sort thingRefs
  showRefThings "Sorted:"
  Run

  when WORKING = true the code works as desired but makes needless copies of the originals.

  
  Thing1 structure:
  Address: 4399796320
  kind:    akRef
  size:    8
  content: (name: "Thing1", word: "CCC", originalAddr: "4399656576")
  
  Thing2 structure:
  Address: 4399796384
  kind:    akRef
  size:    8
  content: (name: "Thing2", word: "BB", originalAddr: "4399656624")
  
  Thing3 structure:
  Address: 4399796448
  kind:    akRef
  size:    8
  content: (name: "Thing3", word: "A", originalAddr: "4399656672")
  
  Reversed:
  4399796448: (name: "Thing3", word: "A", originalAddr: "4399656672")
  4399796384: (name: "Thing2", word: "BB", originalAddr: "4399656624")
  4399796320: (name: "Thing1", word: "CCC", originalAddr: "4399656576")
  
  Sorted:
  4399796320: (name: "Thing1", word: "CCC", originalAddr: "4399656576")
  4399796384: (name: "Thing2", word: "BB", originalAddr: "4399656624")
  4399796448: (name: "Thing3", word: "A", originalAddr: "4399656672")
  Run

  Changing WORKING = false generates the following:

  
  Thing1 structure:
  Address: 4523363968
  kind:    akRef
  size:    8
  content: (name: "Thing1", word: "CCC", originalAddr: "4523363968")
  
  Thing2 structure:
  Address: 4523364016
  kind:    akRef
  size:    8
  content: (name: "Thing2", word: "BB", originalAddr: "4523364016")
  
  Thing3 structure:
  Address: 4523364064
  kind:    akRef
  size:    8
  content: (name: "Thing3", word: "A", originalAddr: "4523364064")
  
  Reversed:
  4523364064: (name: "Thing3", word: "A", originalAddr: "4523364064")
  4523364016: (name: "Thing2", word: "BB", originalAddr: "4523364016\x00ng1\", \x00\x00\x00\x00\x00\x00\x00\n\x00\x00\x00\x00\x00\x00\x004523364064\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n\x00")
  4523363968: (name: "Thing1", word: "CCC", originalAddr: "4523363968\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n\x00\x00\x00\x00\x00\x00\x004523363968\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n\x00")
  
  Sorted:
  4523363968: (name: "Thing1", word: "CCC", originalAddr: "4523363968\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n\x00\x00\x00\x00\x00\x00\x004523363968\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n\x00")
  4523364016: (name: "Thing2", word: "BB", originalAddr: "4523364016\x00ng1\", \x00\x00\x00\x00\x00\x00\x00\n\x00\x00\x00\x00\x00\x00\x004523364064\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n\x00")
  4523364064: (name: "Thing3", word: "A", originalAddr: "4523364064")
  Run

  BTW: I have no idea where the extra null bytes are coming from

  So my question is:

  1. Can ref variables reference non-ref data?

  2. If possible, how do I get safe references to the non-ref data.

     ( I.e. what do I do in the star highlighted code above)

  Hopefully something like this is possible/availalbe as it permits operating on both ref and non-ref data in generic utility methods without having to make unnecessary copies of the originals, while still preserving type safety.

  Thanks for any assistance :-)

---

• zevv (orginal) [2024-02-26T15:31:32+01:00] view original

  Can ref variables reference non-ref data?

  Depends on what you mean with 'reference', you're free to take the address to any data, wherever it lives and use that. But this will not mean Nim will manage the memory and lifetime of the object for you.

  If possible, how do I get safe references to the non-ref data.

  You can't. The only way to create a ref is by using the built in new proc. Check https://nim-lang.org/docs/manual.html#types-reference-and-pointer-types for more details.

---

• DMisener (orginal) [2024-02-26T19:36:51+01:00] view original

  Thanks... re-reading the docs and reading your excellent blog article convinced me to recode existing source to add more ref types so then I won't have to worry about trying to generate safe-references to non-ref types (doing the impossible :-) ). Thanks for the confirmation that "you can't get here from there"

---

• Araq (orginal) [2024-02-27T06:26:46+01:00] view original

  You can also use ptr T and addr to create indirections. Via the =hooks mechanism you can then add automatic storage management.

---