nimforum mirror - Calling proc by address

andrea (orginal) [2015-08-18T18:43:49+02:00] view original

I am trying to make some experiments on distributed computing in Nim. As a first step, I would like to be able to invoke procedures by address. The idea should be that if the same executable is distributed on various machines, the address of functions in memory should be the same, and one could pass a message containing this address and the necessary arguments to ask other machines to execute a specific procedure.

I have followed this example to load and call functions dynamically from a DLL and it works. I have tried to change it in order to call a function in the current executable, using the no-argument version of loadLib like this

import dynlib

proc sayHi*() {.exportc.} =
  echo "hi"

let
  lib = loadLib()
  address = lib.symAddr("sayHi")

echo "The address of sayHi is ", cast[int](address)

Unfortunately it does not seem to be able to locate sayHi and echoes 0.

Is there a way to call a function from the current executable given its name?

In addition, is it possible to get its address given the proc itself? Calling addr on a proc is not valid

Araq (orginal) [2015-08-18T19:34:59+02:00] view original

I don't think it will work at all: https://en.wikipedia.org/wiki/Address_space_layout_randomization

What you need to do is to pass the name of the function as a string and create a name->proc mapping at program startup on your own (mapping["theFunc"] = theFunc). You can use a macro so that you just have to annotate the proc with e.g. .remoteAccess. This is a much cleaner solution anyway, it clearly communicates that something unusual is going on.

andrea (orginal) [2015-08-18T20:42:32+02:00] view original

Thank you. I am aware of ASLR, but I think that it can be disabled programmatically anyway (at least on Linux) by a father process launching the actual executable. In any case, I was also investigating the approach via strings, and it should be easy to generate them with a macro.

The problem that I have is that I am not able to make either approach work. For the address approach, I am not sure how to get the address of a proc, while for the string approach I am not able to get the address of a symbol in the current executable - the example I give above fails to resolve sayHi.

Any ideas how to go on?

Jehan (orginal) [2015-08-18T21:07:37+02:00] view original

I'd have each process use a hash table that maps string handles to procedures. Each process registers all the procedures at startup, and a remote call can then just look them up in the table. Alternatively, you can generate a dispatcher module (using a case statement) at build time; this could then also easily handle different procedure arity and argument types.

andrea (orginal) [2015-08-18T22:26:50+02:00] view original

What you say is doable, but there would be problems of type with the hash table, so I'd favour the second approach. Still, I think one such dispatcher module should essentially be built into the dynlib module. If possible, I'd reuse that feature. If it doesn't work, I think loadLib() without arguments should be removed. Or did I misunderstand something about loadLib?

Araq (orginal) [2015-08-18T22:46:32+02:00] view original

This works for me (Windows). Dunno if it works on Linux, I think it does.

import dynlib

proc sayHi*() {.exportc, dynlib.} =
  echo "hi"

let
  lib = loadLib()
  address = lib.symAddr("sayHi")

echo "The address of sayHi is ", cast[int](address)

Jehan (orginal) [2015-08-19T00:23:23+02:00] view original

You can also go with a hash table if you use macros to generate wrappers that convert serialized argument lists into the actual arguments so that every such wrapper has the same type.

andrea (orginal) [2015-08-19T09:15:36+02:00] view original

You are right, that my be the simplest approach, considering that I will have to use a macro anyway to store a function id into a hash table.

Anyway, I tried the example by Araq (I was missing a dinlyb pragma) on Ubuntu and it does not work for me - still reports address 0. Should I compile the executable with some particular flag? Just curious, I think I will go with the hashtable approach anyway

andrea (orginal) [2015-08-19T11:07:10+02:00] view original

I tried to go with the hashmap approach but it seems not completely trivial. What I would like to do is to generate a table at compile time where string identifiers are mapped to procs - for simplicity let me take void -> void procs. My first attempt goes like

static:
  var
    count = 0
    procMap = newTable[string, proc()]()

proc onNodeSend(n, procId: string) =
  # send the actual request to node n

macro onNode(n: string, f: proc): untyped =
  template inner(n, id) =
    onNodeSend(n, id)
  
  let fId = "func_" & $(count)
  # procMap[fId] = f
  count += 1
  result = getAst(inner(n, fId))

to be used like this

onNode "2", proc() =
  echo "Hello from node 2"

but I run into two issues.

First, I cannot assign f to the table (the commented line does not compile) because inside the macro f is a NimNode representing a proc and not a proc per se

Second, I need the table to exist at runtime (of course!) but in order to add to it, I need it to exist at compile time, which is why I put it into a static block. What I would need is a way to tell: ok, I am finished, please transform this table (which is a var into the static scope) into a const in the usual scope.

andrea (orginal) [2015-08-21T11:20:09+02:00] view original

Turns out the simplest solution here is to just call closures and rely on the address returned by f.rawProc (which is consistent across process, regardless of having ASLR activated - I think ASLR ony causes issues for DLLs).

The problem that I have now is to properly send a captured environment because I don't know how to get its size

Mirror of forum.nim-lang.org

1560 :: Calling proc by address