Mirror of forum.nim-lang.org

5018 :: Problem wrapping a shared library with Nimterop (Sundials-N_Vector)

• hugogranstrom (orginal) [2019-07-16T15:47:38+02:00] view original

  I have started to wrap the Sundials project and my first goal was to wrap the N_Vector shared library, which I to some extent has succeeded with but in a rather hacky, and not so seamless, way. The structure of the files can be found in my github repo: https://github.com/HugoGranstrom/nimsundials. The headers are in the include/ folder and the shared library is in the lib/ folder. The file I'm writing all the code below in is located at the root of the repo and is called nvector.nim. I wondered if there is another way of doing it. I have tried to solve it in various ways which are shown below, and only the last one worked.

  The only thing that differs between the different versions are how the headers are imported so this is the main structure of the file:

  import os, strformat
  
  import nimterop/[cimport, paths]
  
  const srcDir = currentSourcePath().parentDir()
  const inclDir = srcDir/"include"
  const libDir = srcDir/"lib"
  
  
  static:
      cDebug()
      cDisableCaching()
  
  cIncludeDir(srcDir/"include")
  cIncludeDir(srcDir/"include/nvector")
  cIncludeDir(srcDir/"include/sundials")
  cIncludeDir(srcDir/"lib")
  
  cPlugin:
      import strutils
      proc onSymbol*(sym: var Symbol) {.exportc, dynlib.} =
          if sym.kind == nskType:
              if sym.name == "_N_VectorContent_Serial":
                  sym.name = "N_VectorContent_Serial_Base"
          sym.name = sym.name.strip(chars = {'_'})
  
  
  const
      libsundials_nvecserial = libDir/"libsundials_nvecserial.dll"
  
  #######################
  #######################
  
  Insert the different codes here
  
  #######################
  #######################
  
  
  type
      NVectorType* = object
          length*: int
          rawVector*: ref[N_Vector]
  
  template ptr2Array[T](p: pointer): auto = cast[ptr UncheckedArray[T]](p)
  template array2Ptr[T](arr: openArray[T]): auto = addr(arr[0])
  
  template `->`(a, b: untyped): untyped =
      a[].b
  
  template NV_CONTENT_S(v: untyped): untyped = cast[N_VectorContent_Serial](v->content)
  template NV_LENGTH_S(v: untyped): untyped = NV_CONTENT_S(v) -> length
  template NV_OWN_DATA_S(v: untyped): untyped = NV_CONTENT_S(v) -> own_data
  template NV_DATA_S(v: untyped): untyped = NV_CONTENT_S(v) -> data
  template NV_Ith_S(v: untyped, i: sunindextype): untyped = ptr2Array[realtype](NV_DATA_S(v))[i]
  
  proc newNVector*(length: int): NVectorType =
      if length <= 0:
          raise newException(ValueError, "NVector length must be greater than 0")
      result.length = length
      result.rawVector = new N_Vector
      result.rawVector[] = N_VNew_Serial(result.length)
  
  var v = newNVector(3)
  echo NV_LENGTH_S(v.rawVector[])
  Run

  Method 1:

  cImport(inclDir/"nvector/nvector_serial.h", dynlib = "libsundials_nvecserial", recurse = true)
  Run

  Here I try to link the dynamic library and recurse on it's #include-ed headers. When I try to compile this it prints out the produced Nim code but then it freezes here. I left it for an hour but nothing had happened:

  
  CC: stdlib_system.nim
  CC: nvector.nim
  
  Run

  Method 2:

  cImport(inclDir/"sundials/sundials_config.h")
  cImport(inclDir/"sundials/sundials_types.h")
  cImport(inclDir/"sundials/sundials_nvector.h")
  
  cImport(inclDir/"nvector/nvector_serial.h", dynlib = "libsundials_nvecserial")
  Run

  Now I manually import the headers it needs. Now I instead get this error:

  CC: stdlib_system.nim
  CC: nvector.nim
  Error: execution of an external compiler program 'gcc.exe -c  -w -mno-ms-bitfields -DWIN32_LEAN_AND_MEAN -IC:/Users/elev/code/nimsundials/include -IC:/Users/elev/code/nimsundials/include/nvector -IC:/Users/elev/code/nimsundials/include/sundials -IC:/Users/elev/code/nimsundials/lib  -IC:\Users\elev\.choosenim\toolchains\nim-0.20.0\lib -IC:\Users\elev\code\nimsundials -o C:\Users\elev\nimcache\nvector_d\nvector.nim.c.o C:\Users\elev\nimcache\nvector_d\nvector.nim.c' failed with exit code: 1
  
  C:\Users\elev\nimcache\nvector_d\nvector.nim.c: In function 'NimMainModule':
  C:\Users\elev\nimcache\nvector_d\nvector.nim.c:533:26: error: dereferencing pointer to incomplete type 'struct _N_VectorContent_Serial'
    T1_[0] = nimInt64ToStr((*((struct _N_VectorContent_Serial*) ((*(*v_0Jd0gsgUEmSn2jVcFETIqw.rawVector)).content))).length);
                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  Run

  It seems to complain that it can't find the type 'struct _N_VectorContent_Serial'. This is the Nim code that is generated for it (I renamed it to N_VectorContent_Serial_Base in Nim otherwise they would have the same name.):

  N_VectorContent_Serial_Base* {.importc: "struct _N_VectorContent_Serial", bycopy.} = object
      length*: sunindextype
      own_data*: cint
      data*: ptr realtype
  N_VectorContent_Serial* {.impnvector_serial.} = ptr N_VectorContent_Serial_Base
  Run

  This original header for this is the file nvector_serial.h which is the one that is linked to the shared library. There it looks like this:

  struct _N_VectorContent_Serial {
    sunindextype length;   /* vector length       */
    booleantype own_data;  /* data ownership flag */
    realtype *data;        /* data array          */
  };
  
  typedef struct _N_VectorContent_Serial *N_VectorContent_Serial;
  Run

  Method 3:

  cImport(inclDir/"sundials/sundials_config.h")
  cImport(inclDir/"sundials/sundials_types.h")
  cImport(inclDir/"sundials/sundials_nvector.h")
  
  cImport(inclDir/"nvector/nvector_serial2.h")
  cImport(inclDir/"nvector/nvector_serial.h", dynlib = "libsundials_nvecserial")
  Run

  The way I solved this was by commenting out the type definition in the C header and copy it to a separate header called nvector_serial2.h which I imported before the main header. Now the code compiles and runs. I compared the generated C code before and after my hacky trick and it seems that when I moved the type definitions to a separate file it was #included in the C code, but before it was not.

  I know this may be a bit hard to follow but the Sundials code base is it as well :/

  Has anyone else had similar experiences or am I just missing something obvious?

---