nimforum mirror

rozovr (orginal) [2015-05-29T16:08:35+02:00] view original

I am utterly failing to interface with C++ code I need. I've tried to follow examples from the manual, tried to create something useable with c2nim, and looked at the example I found through the forum: https://github.com/3dicc/Urhonimo - all leading to much frustration.

The code I need is a Bloom filter library from here: http://minia.genouest.org/files/minia-1.6906.tar.gz I am aware of the bloom module in nim and it's very nice, but I need to use the above because my purpose is to interface further with the rest of the code there in such a way that the specific Bloom filter and the false positives resulting from it are essential.

I've pasted the two files Bloom.cpp and Bloom.h here: http://pastebin.com/8iwzU2Gp, and here: http://pastebin.com/LLrc28TZ

When I try c2nim --cpp with either, it reports syntax errors, but I know this code works. Regarding integrating it manually into a nim file, My feeling is I don't know where to begin. It's not clear to me if my aim should be be linking to a compiled object file as in the "sloppy" example in the manual, using a {.compile.} pragma as the nim bloom library uses to get murmurhash, or if it's something else entirely.

I would really really appreciate basic assistance to get started - to the point of being able to create a BF, add to it, and query it. Many thanks, Roye

tamtaradam (orginal) [2015-05-29T16:56:10+02:00] view original

I have the similar problem. And noone didn`t say me how work with c2nim. I found a good article. But that dosent helpful for me. http://goran.krampe.se/2014/10/16/nim-wrapping-c/ I hope, that we can solve this problem together. I alredy tried use c2nim with 2 other libs: FLTK and gtkmm. When i try to convert a main header file of gtkmm to .nim, c2nim ignore all includes. When i try to convert a FLTK headers, c2nim says, that files have a silly syntax errors. But i know: it code works. Any idea?

novist (orginal) [2015-05-29T16:59:24+02:00] view original

c2nim cant parse entire c/c++. If you are getting syntax errors then edit headers until c2nim eats it. thats how they do it with Urho3D.

tamtaradam (orginal) [2015-05-29T17:05:48+02:00] view original

Thanks. I not pro in c++. Errors, that get c2nim are mystical for me. For example:

FL_EXPORT void gl_start();

FL_EXPORT void gl_finish();

In first line c2nim get this:

(64, 25) Error: ';' expected

I dont know, what is wrong. Its so strange.

Varriount (orginal) [2015-05-29T17:37:38+02:00] view original

@tamtaradam FL_EXPORT is likely a macro used to simplify function declaration. You'll either need to remove it, or replace it's #define pragma with #def

rozovr (orginal) [2015-05-29T18:10:10+02:00] view original

I'm seeing similar errors to ';' expected, or identifier expected. I have no problem cutting out lines until something changes but I don't know how to gauge when I've improved my situation or not. For example, I'm not sure if c2nim is supposed to be run on .cpp files or headers. I also don't know how further dependencies get found, and the list goes on. Novist, can you clarify what you meant by "that's how they do with it Urho3D"? I have trouble believing it's trial and error like I'm doing.

tamtaradam (orginal) [2015-05-29T18:39:06+02:00] view original

I got a funny situation. I create some fixes in headers. I just want to run c2nim and when i start it my computer... get.. very... slow... and get frozen. I restart it and try to run again. I have time to stop it, and i noticed, that c2nim eat all my RAM! It`s time to find mem leaks in c2nim code :) But what i will do, i don`t know...

Varriount (orginal) [2015-05-29T19:13:18+02:00] view original

@tamtaradam What kind of files are you feeding into c2nim?

tamtaradam (orginal) [2015-05-29T19:27:33+02:00] view original

@Varriount Simply header c++ file, nothing uncanny. http://pastebin.com/6KAEe9pJ

jlp765 (orginal) [2015-05-30T01:21:57+02:00] view original

c2nim currently doesn't handle empty defines, like the following

#define FL_EXPORT  /* empty definition */

FL_EXPORT void gl_start();

Replace #define with #def (as mentioned above) is your friend ;-)

Any included header files you want to become nim imports like "import ...", must be quoted, so convert <FL/FL_ints.h> to "FL/FL_ints.h"

you probably will want a standard addition to each header file, like the following:

#ifdef C2NIM
  #  dynlib libfltk
  #  cdecl
  #  if defined(windows)
  #    define libfltk "fltk.dll"
  #  elif defined(macosx)
  #    define libfltk "libfltk.dylib"
  #  else
  #    define libfltk "libfltk.so"
  #  endif
  #  mangle uint8_t uint8
  #  mangle uint16_t uint16
  #  mangle uint32_t uint32
  #  mangle uint64_t uint64
  #endif

rozovr (orginal) [2015-05-30T15:15:04+02:00] view original

here's the resulting nim file:


#
#  Bloom.h
#
#  Created by Guillaume Rizk on 9/02/12.
#

when not defined(Bloom_h):
  const
    Bloom_h* = true
  
  when not defined(kmer_type):
    type
      bloom_elem* = uint64
  else:
    type
      bloom_elem* = kmer_type
const
  NSEEDSBLOOM* = 10
  CUSTOMSIZE* = 1

var bits_per_char*: cint = 0x00000008

# 8 bits in 1 char(unsigned)

var bit_mask*: array[bits_per_char, cuchar] = [0x00000001, 0x00000002,
    0x00000004, 0x00000008, 0x00000010, 0x00000020, 0x00000040, 0x00000080]

var cpt_per_char*: cint = 2

var cpt_mask*: array[cpt_per_char, cuchar] = [0x0000000F, 0x000000F0]

var cpt_mask21*: array[21, uint64] = [0x0000000000000007'i64,
                                      0x0000000000000038'i64,
                                      0x00000000000001C0'i64,
                                      0x0000000000000E00'i64,
                                      0x0000000000007000'i64,
                                      0x0000000000038000'i64,
                                      0x00000000001C0000'i64,
                                      0x0000000000E00000'i64,
                                      0x0000000007000000'i64,
                                      0x0000000038000000'i64,
                                      0x00000001C0000000'i64,
                                      0x0000000E00000000'i64,
                                      0x0000007000000000'i64,
                                      0x0000038000000000'i64,
                                      0x00001C0000000000'i64,
                                      0x0000E00000000000'i64,
                                      0x0007000000000000'i64,
                                      0x0038000000000000'i64,
                                      0x01C0000000000000'i64,
                                      0x0E00000000000000'i64,
                                      0x7000000000000000'i64]

var cpt_mask32*: array[32, uint64] = [0x0000000000000003'i64,
                                      0x000000000000000C'i64,
                                      0x0000000000000030'i64,
                                      0x00000000000000C0'i64,
                                      0x0000000000000300'i64,
                                      0x0000000000000C00'i64,
                                      0x0000000000003000'i64,
                                      0x000000000000C000'i64,
                                      0x0000000000030000'i64,
                                      0x00000000000C0000'i64,
                                      0x0000000000300000'i64,
                                      0x0000000000C00000'i64,
                                      0x0000000003000000'i64,
                                      0x000000000C000000'i64,
                                      0x0000000030000000'i64,
                                      0x00000000C0000000'i64,
                                      0x0000000300000000'i64,
                                      0x0000000C00000000'i64,
                                      0x0000003000000000'i64,
                                      0x000000C000000000'i64,
                                      0x0000030000000000'i64,
                                      0x00000C0000000000'i64,
                                      0x0000300000000000'i64,
                                      0x0000C00000000000'i64,
                                      0x0003000000000000'i64,
                                      0x000C000000000000'i64,
                                      0x0030000000000000'i64,
                                      0x00C0000000000000'i64,
                                      0x0300000000000000'i64,
                                      0x0C00000000000000'i64,
                                      0x3000000000000000'i64,
                                      0xC000000000000000'i64]

# static const unsigned char incr_cpt_table[2][255] =
# {
# {1, 2,3},
# {3, 4,3},
# };
#

var rbase*: array[NSEEDSBLOOM, uint64] = [0xAAAAAAAA55555555'i64,
    0x33333333CCCCCCCC'i64, 0x6666666699999999'i64, 0xB5B5B5B54B4B4B4B'i64,
    0xAA55AA5555335533'i64, 0x33CC33CCCC66CC66'i64, 0x6699669999B599B5'i64,
    0xB54BB54B4BAA4BAA'i64, 0xAA33AA3355CC55CC'i64, 0x33663366CC99CC99'i64]

#
#
# 0x2E7E5A8996F99AA5,
# 0x74B2E1FB222EFD24,
# 0x8BBE030F6704DC29,
# 0x6D8FD7E91C11A014,
# 0xFC77642FF9C4CE8C,
# 0x318FA6E7C040D23D,
# 0xF874B1720CF914D5,
# 0xC569F575CDB2A091,
#
#static uint64_t pri1=0x5AF3107A401FULL;
#static uint64_t pri2 =0x78C27CE77ULL;

type
  Bloom* = object # #ifdef _largeint
                  #     inline uint64_t hash_func(LargeInt<KMER_PRECISION> elem, int num_hash);
                  # #endif
                  # #ifdef _ttmath
                  #     inline uint64_t hash_func(ttmath::UInt<KMER_PRECISION> elem, int num_hash);
                  # #endif
                  # #ifdef LP64
                  #     inline uint64_t hash_func(__uint128_t key, int num_hash);
                  # #endif
    user_seed*: uint64
    seed_tab*: array[NSEEDSBLOOM, uint64]
    n_hash_func*: cint
    nchar*: uint64
    blooma*: ptr cuchar
    tai*: uint64
    nb_elem*: uint64


proc setSeed*(this: var Bloom; seed: uint64)
proc set_number_of_hash_func*(this: var Bloom; i: cint)
proc add*(this: var Bloom; elem: bloom_elem)
proc contains*(this: var Bloom; elem: bloom_elem): cint
proc dump*(this: var Bloom; filename: cstring)
proc load*(this: var Bloom; filename: cstring)
proc weight*(this: var Bloom): clong
proc constructBloom*(tai_bloom: cint): Bloom {.constructor.}
proc constructBloom*(tai_bloom: uint64): Bloom {.constructor.}
proc constructBloom*(): Bloom {.constructor.}
proc destroyBloom*(this: var Bloom)
type
  BloomCpt* = object of Bloom


proc constructBloomCpt*(tai_bloom: cint): BloomCpt {.constructor.}
proc constructBloomCpt*(): BloomCpt {.constructor.}
proc add*(this: var BloomCpt; elem: bloom_elem)
proc contains_n_occ*(this: var BloomCpt; elem: bloom_elem; nks: cint): cint
type
  BloomCpt3* = object of BloomCpt
    blooma3*: ptr uint64


proc constructBloomCpt3*(tai_bloom: cint): BloomCpt3 {.constructor.}
proc destroyBloomCpt3*(this: var BloomCpt3)
proc add*(this: var BloomCpt3; elem: bloom_elem)
proc contains_n_occ*(this: var BloomCpt3; elem: bloom_elem; nks: cint): cint
type
  BloomCpt2* = object of BloomCpt
    blooma2*: ptr uint64


proc constructBloomCpt2*(tai_bloom: cint): BloomCpt2 {.constructor.}
proc destroyBloomCpt2*(this: var BloomCpt2)
proc add*(this: var BloomCpt2; elem: bloom_elem)
proc contains_n_occ*(this: var BloomCpt2; elem: bloom_elem; nks: cint): cint

@tamtaradam, I skipped the dynlib stuff but used the mangle lines @jlp765 suggested

tamtaradam (orginal) [2015-06-01T17:02:49+02:00] view original

Okay, i tried to fix header files. At my look its clear, but c2nim all tha same get frozen, when i try to convert this file. Fixed file http://pastebin.com/znuYphEe I comment all includes for check, when c2nim get frozen. Its so strange. C2NIM dont get any error, its simply get frozen...

rozovr (orginal) [2015-06-01T23:17:11+02:00] view original

can't say I have any more clues...

jlp765 (orginal) [2015-06-02T00:49:58+02:00] view original

@tamtaradam: The following works (you want define not def where the macro doesn't require expanding)

#ifdef WIN32
# ifdef _WIN64
#  define FL_SOCKET uint64_t
# else
#  define FL_SOCKET int
# endif
#else
# define FL_SOCKET int
#endif

which allows you to find the next problem line is

typedef void (Fl_Label_Draw_F)(const Fl_Label *label, int x, int y, int w, int h, Fl_Align align);

but this will compile

typedef void (*Fl_Label_Draw_F)(const Fl_Label *label, int x, int y, int w, int h, Fl_Align align);

Of course you then have to check that what is generated is what is actually required ;-) and can also compile

rozovr (orginal) [2015-06-02T06:45:49+02:00] view original

@jlp765, any chance you can help with my earlier question? I'm referring to what to do once I get a header file processed with c2nim: how do I use that along with my .cpp file incuding the actual function implementations? E.g., with the .nim I posted above I eventually get the compiler errors that implementations of the procs are expected, but implementing them again defeats the purpose of using the C++ library.

Varriount (orginal) [2015-06-02T08:13:42+02:00] view original

You need to mark the processed procedures with the appropriate FFI pragmas (in particular, the importc and dynlib pragmas). This tells Nim to retrieve procedure implementations from headers, libraries,etc.

jlp765 (orginal) [2015-06-02T12:25:26+02:00] view original

@rozovr,

I'm no c++ c2nim guru, so maybe others can help out here.

The header file you included had public and private sections, but from the perspective of Nim calling the c++ library, only the public parts are required

I couldn't get the class to compile as it was.

I tried

typedef class Fl_Widget_Tracker {
  int deleted() ;
  int exists() ;
  void Fl_Widget_Tracker(Fl_Widget *wi) ;
  Fl_Widget *widget(void) ;
} ;

which produces

type
  Fl_Widget_Tracker* = object
    deleted*: proc (): cint {.cdecl.}
    exists*: proc (): cint {.cdecl.}
    Fl_Widget_Tracker*: proc (wi: ptr Fl_Widget) {.cdecl.}
    widget*: proc (): ptr Fl_Widget {.cdecl.}

but this result isn't correct, so hopefully someone else knows what to do.

The overall process as I understand it is:

I assume you have a shared library/dll (or have compiled the c++ code to a shared lib/dll), and have that library located somewhere that can be found by Nim (usually nim/bin directory?)

The c2nim define statements (that you add at the top of the header file) cause the necessary FFI pragmas to be added to each call to the library/dll that are in the header files.

(as I understand it, you don't need to compile all you c++ cpp files, only the header files that define the interface to the lib/dll, and that becomes the import nim file(s) that specify how to interface from nim to the lib/dll - refer to the last dot point).

c2nim produces nim files

Compile each nim file made by c2nim starting firstly with the nim files that are imports to your other nim files (and fix up the header files and run c2nim and recompile the nim files, ad nausium, ad infinitum) until the nim files all compile, or until you have enough compiled to then do the next dot point.

you may want to combine multiple c2nim header files or nim output files into a single nim file to provide a single import file to be called in the next dot point (which I refer to below as import whateveryourlibnameis )

Write a test program to test that you can successfully call the shared library/dll. Your test nim program will contain import whateveryourlibnameis.

Then try and chase down any further interfacing problems (wrong calling conventions, wrong sizes of types, wrong ptr/value passing, ....), and test on multiple platforms.

now go back and make sure that all the calls to the lib/dll are defined in your c2nim nim files

write a nimble package and put it on github

now brag (I mean avdertise) the existence of the nimble library (and find out someone else has already done it ;-) )

Have a look at the official and unofficial nimble packages to see what some other libraries look like, and try and reverse engineer a similar result ;-)

Araq (orginal) [2015-06-02T14:31:52+02:00] view original

I think the major problem is that typedef class Fl_Widget_Tracker is too uncommon so it was never implemented properly. Use class Fl_Widget_Tracker instead.

rozovr (orginal) [2015-06-02T18:10:00+02:00] view original

Thanks again guys. I'll give it another shot. I think we were talking about different files since @tamtaradam posted a different set after me, but I assume most of your advice applies in my case as well.

Mirror of forum.nim-lang.org

1272 :: C++ FFI