Mirror of forum.nim-lang.org

11172 :: Problem with closures and ORC

• lou15b (orginal) [2024-03-06T22:29:19+01:00] view original

  I encountered the following problem(s) while implementing destructors in the code base I'm working on. Nim version is 2.0.2, compiled at debug level.

  The first file defines the types used in the code example to follow:

  # types.nim
  
  # ############ For Section A ############################
  import std/intsets
  
  type
    SelectorObj = object of RootObj
      onSelectionChange*: proc()
      selections*: IntSet
    
    Selector* = ref SelectorObj
  
  proc `=destroy`*(v: SelectorObj) =
    echo "#### Entering SelectorObj destructor"
    try:
      `=destroy`(v.onSelectionChange.addr[])
    except:
      echo "Fault raised destroying onSelectionChange"
    
    try:
      `=destroy`(v.selections.addr[])
    except:
      echo "Fault raised destroying selections"
  
  proc init*(v: Selector) =
    v.selections = initIntSet()
  
  proc newSelector*(): Selector =
    result = Selector.new()
    result.init()
  # ########################################
  
  # ############ For Section B ############################
  type Test* = object
    code : proc()
    testName: string
  
  proc `=destroy`(x: Test) =
    echo "Entering Test destructor"
    echo "\t testName: ", x.testName
    if not isNil(x.code):
      try:
        `=destroy`(x.code.addr[])
      except:
        echo "Fault raised destroying Test closure"
    `=destroy`(x.testName)
  
  proc makeTest*(code: proc(), testName: string): Test =
    result.code = code
    result.testName = testName
  
  # ==========
  type
    ContextObj* = object of RootObj
      name*: string
    Context* = ref ContextObj
  
  proc `=destroy`*(w: ContextObj) =
    echo "#### Entering ContextObj destructor"
    echo "####\t Context name: ", w.name
    `=destroy`(w.name)
  
  proc init*(w: Context, name: string) =
    w.name = name
  
  proc newContext*(name: string): Context =
    result.new()
    result.init(name)
  # ########################################
  Run

  Note that the expression pattern

  `=destroy`(x.xxx.addr[])
  Run

  is my best guess on how to properly clean up objects that do not have destructors defined. That would includes closures as well as types imported from an external library.

  If you know better, please let me know.

  Now to the matter at hand. The following example demonstrates the issue(s) I encountered.

  # main.nim
  
  import ./types
  import std/intsets
  
  proc testproc(ctx: Context) = discard
  
  proc main() =
    echo "Entered main()"
    
    # ######### Section A #########
    let selector = newSelector()
    let csel {.cursor, used.} = selector
    
    selector.onSelectionChange = proc() =
      var selections: seq[int] = @[]
      # for item in items(csel.selections):   # Works okay for this
      for item in items(selector.selections):   # Does not work for this
        selections.add(item)
    # ##############################
    
    # ######### Section B #########
    let context = newContext("context")
    
    let generalTest {.used.} =
      makeTest(proc () = testproc(context), "test #1")
    # ##############################
    
    echo "Leaving main()"
  
  echo "\nCalling main()"
  main()
  echo "Returned from main()"
  echo "...Done\n"
  Run

  Executing the above code gives the following output:

  
  Calling main()
  Entered main()
  Leaving main()
  Entering Test destructor
      testName: test #1
  Returned from main()
  ...Done
  Run

  Note that there is no indication that the destructor is called for the selector object. Nor is the destructor called for the context object - even though the destructor was called for the containing generalTest object.

  Using a cursor instead of a regular reference in the for item in ... statement (i.e. replacing selector.selections by csel.selections) gives:

  
  Calling main()
  Entered main()
  Leaving main()
  Entering Test destructor
       testName: test #1
  #### Entering SelectorObj destructor
  #### Entering ContextObj destructor
  ####     Context name: context
  Returned from main()
  ...Done
  Run

  Note that now the destructor for both selector and context have been called. Everything looks fine.

  The fact that the issue is fixed using a cursor instead of a normal reference (thereby breaking a reference cycle) tells me that there is something about it that the ORC cycle collector can't currently handle.

  I have the following questions:

  • Is this a bug in the cycle collector, or is it an unfixable limitation?
  • If the latter, can the problem be detected and reported, at least for debug loads?
  • Why does the cycle collector's failure to clean up stuff from Section A in the code impact the cleanup of stuff from Section B? The only thing they have in common is the fact that they both use closures - which are entirely separate from each other.

---

• sls1005 (orginal) [2024-03-07T10:33:44+01:00] view original

  The simplest way I know to destroy a closure is to assign it with nil. But this, of course, requires the variable to be mutable.

  I think the issue here is that the closure catchs the selector while being a field of it at the same time, so they reference each other and thus form a cycle?

  If so, I think you may set the field of selector to nil within the main function, so as to manually break the cycle. I don't know if there's a better way or if it has something to do with section B. Have you ever tried to compile the code with --mm:refc?

---

• Araq (orginal) [2024-03-07T12:08:39+01:00] view original

  You need to call GC_fullCollect to force the cycle collector's hand. Otherwise it runs when it thinks is wise and "at program exit" does not count as the OS cleans up things anyway.

---

• lou15b (orginal) [2024-03-07T15:10:21+01:00] view original

  Thanks!

  Calling GC_fullCollect just before program exit fixed the problem.

  This is important for my efforts using valgrind to identify and fix any leaks in the code base I'm working on - I don't want uncollected cycles hanging around to confuse things.

---