Mirror of forum.nim-lang.org

10074 :: Calling Nim proc from C and handling the cstring (char array) returned

• olwa (orginal) [2023-04-05T16:47:19+02:00] view original

  I want to call a Nim proc from C which returns a cstring, the same as the "gimme" example at https://nim-lang.org/docs/backends.html#memory-management-strings-and-c-strings. I want to check that the way I am using the char array returned to C is sound.

  This is a minimal example of my current approach, where the contents of the cstring returned by Nim are copied over to C-managed memory for further use.

  In gimme.nim (following this):

  import std/random
  
  proc gimme(): cstring {.exportc.} =
    result = "Hey there C code! " & $rand(100)
  Run

  In gimme_wrapped.c:

  #include <stdio.h>
  #include <string.h>
  #define BUFFSIZE 1024
  #define MIN(x,y) (((x) <= (y)) ? (x) : (y))
  
  extern char* gimme(void);
  extern void NimMain();
  
  void gimme_wrapper(char* output, int maxlength) {
    NimMain();
    char* data_from_nim = gimme();
    // No more calls to nim till output is copied:
    for (int i=0; i<MIN(strlen(data_from_nim), maxlength); i++) {
      output[i] = data_from_nim[i];
    }
  }
  
  int main(void) {
    char output[BUFFSIZE] = "";
    gimme_wrapper(output, BUFFSIZE);
    printf("Data passed to C from Nim:\n%s\n", output);
    return 0;
  }
  Run

  Set $NIM_LIB_PATH and build with:

  
  nim c --noMain --noLinking gimme.nim
  gcc -o gimme_wrapped -I$HOME/.cache/nim/gimme_d -I$NIM_LIB_PATH $HOME/.cache/nim/gimme_d/*.c gimme_wrapped.c
  Run

  This apparently works OK and prints the expected result. I assume that as long as I make no more calls to Nim between the call to gimme and the C code that copies its output, Nim's garbage collector can be relied on not to free the memory holding the data of interest. My understanding is that it is only further calls to Nim resulting in memory allocation by Nim that will trigger garbage collection and result in possible corruption of the data.

  I wanted to check that my assumptions are correct and that this is a reasonable approach. In particular this is because a slightly earlier version of the documentation contained a description of this approach which has since been cut ("However, from a practical standpoint..." at https://nim-lang.org/1.6.0/backends.html#memory-management-strings-and-c-strings). Are there known circumstances where it could fail, or perhaps future plans that mean it should not be relied on? The use of GC_ref and GC_unref seems straightforward when calling C code from Nim, but less so when calling Nim from C.

---

• Araq (orginal) [2023-04-05T20:40:53+02:00] view original

  proc gimme(): cstring {.exportc.} =
    result = "Hey there C code! " & $rand(100)
  
  Run

  is broken. Here is a sketch of a solution:

  proc gimme(): cstring {.exportc.} =
    let tmp = "Hey there C code! " & $rand(100)
    result = cast[cstring](malloc(tmp.len + 1))
    copyMem(result, tmp, tmp.len + 1)
  
  # C caller must call `free`
  
  Run

  Your code sort of used to work with older GCs as you noticed but it doesn't work with ORC anymore.

  If you can avoid having your main in C, do so immediately. And then throw the rest of the C code away too. ;-)

---

• sls1005 (orginal) [2023-04-06T05:35:48+02:00] view original

  cstrings are untraced pointers. They cannot hold the ownership of a Nim string. When you assign "Hey there C code! " & $rand(100) (a string as a temporary object) to result (a cstring), the string's life ends because its reference count is reduced to zero and is not increased, and as a result, it could be freed at any time. Your code might work because heap data might not be erased frequently, but it's still considered used-after-free.

---

• olwa (orginal) [2023-04-07T09:40:22+02:00] view original

  Thanks, that's very helpful. When I explicitly allocate in Nim and then free from C then my code fails with "pointer being freed was not allocated". But handling the memory entirely from C and making it available to Nim via an argument works fine for me (that is, have Nim copy its output to C-provided memory instead of returning its output):

  void gimme_wrapper(char* output, int maxlength) {
    NimMain();
    gimme(output, maxlength);
  }
  Run

  proc gimme(output: cstring, maxlength: int) {.exportc.} =
    let tmp = "Hey there C code! " & $rand(100)
    copyMem(output, tmp.cstring(), tmp.len + 1) # TODO: truncate tmp to maxlength
  Run

---