Mirror of forum.nim-lang.org

10741 :: Dealing with arrays (of bytes) across library boundaries, specifically C#

• Boston (orginal) [2023-12-06T22:43:46+01:00] view original

  I just spent the day figuring out how to Marshal arrays of bytes between C# and Nim (C# calling a DLL written in Nim) and wanted to share what I came up with can give people the chance to correct me. For some context, this is a Nim library that can be used natively or compiled to a DLL, the code presented will be an exported DLL function.

  One of the Nim exports I have written looks like this:

  proc transmitAndReceive(i2cAddress: byte, txdata: ptr UncheckedArray[byte], rxdata: var ptr UncheckedArray[byte], txlen, rxlen: int ): MyEnum {.exportc, dynlib, stdcall.} =
      let convertedTx = @(toOpenArray(txdata, 0, txlen))
      var convertedRx = newSeq[byte](rxlen)
      result = myModule.transmitAndReceive(i2cAddress, convertedTx, convertedRx)
      if rxlen > 0 and convertedRx.len >= rxlen:
        rxdata = cast[ptr UncheckedArray[byte]](CoTaskMemAlloc(rxlen))
        copyMem(rxdata, convertedRx[0].addr, rxlen)
  Run

  • For the unfamiliar I believe CoTaskMemAlloc is how .net allocs and frees for "unmanaged" code and possibly for managed code.
  • CoTaskMemAlloc WILL alloc a 0 length block and return a ptr to it if its argument is 0.
  • Returns a pointer to a block of memory if successful and null if unsuccessful.

  proc CoTaskMemAlloc(n: int): pointer {.importc, dynlib:"ole32.dll".}'

  The C# code for using the DLL:

  
  [DllImport(dllname, EntryPoint = "transmitAndReceive", CallingConvention = CallingConvention.StdCall)]
  public static extern MyEnum TransmitAndReceive(
    byte i2cAddress,
    [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.U8, SizeParamIndex = 3)] byte[] txData,
    [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.U8, SizeParamIndex = 4)] out byte[] rxData,
    int txlen, int rxlen);
  Run

  • Yes StdCall is the default calling convention (Winapi) but I prefer to specify it.
  • I have to specify the EntryPoint because of style reasons, I could likely make a macro that fixes this automatically but it changes from language to language anyway.
  • The Marshal class tells the program how to pass data around, how to Marshal it. In this example I am saying that txData and rxData are C style arrays, which if im not wrong is the same as Nim's ptr UncheckedArray[T] and ArraySubType declares the type of the values of the array, in this case an unsigned int8, SizeParamIndex tells the Marshaller what field of the call indicates the length of the array, here that is txlen and rxlen.
  • The out keyword indicates that the called code is responsible for initialization.

  So how do I use this?

  
  byte[] getacq = new byte[] { 3 }; // Tells the thing I'm talking to, to return the data from channel 3
  byte[] recvacq; // Where I will receive the data, Null here because no initialization or assignment.
  TransmitAndReceive(0x42, getacq, out recvacq, getacq.Length, 5);
  Run

  • 0x42 is the device address.

  I pass the data I want to send (getacq) which is then Marshalled as a C style array that has length getacq.length (1). My Nim DLL then uses that information to create a Seq that is processed and transmitted, after all that happens the DLL allocs a new block of memory the size of rxlen, then copies that data from convertedRx which contains the response from the device in question. This should be safe because the Seq I make is of length rxlen as is the block of memory I alloc, so copyMem can not read past or overflow and I do not alloc if convertedRx got smaller in the call chain. (There is code missing that does Null checking for the alloc and returns an error). Once the call returns the Marshaller will convert the C style array to a .net array using the SizeParamIndex and recvacq becomes a valid not Null array, in the case of this code and array of len 5 with the first byte being a status and the next 4 a float32. Additionally when deemed appropriate .net will dealloc the array which I accidentally proved to myself with a typo.

---