nimforum mirror - simple event system

enthus1ast (orginal) [2022-01-08T12:23:26+01:00] view original

I'm trying to write a simple event system (for use in my ECS and game).

I want to be able to connect and disconnect events at runtime.

This is what i have so far:

import tables, sets, hashes
type
  Reg = ref object
    ev: Table[Hash, HashSet[pointer]]

proc connect(reg: Reg, ty: typedesc, cb: pointer) =
  const typehash = hash($ty.type)
  if not reg.ev.hasKey(typehash):
    reg.ev[typehash] = initHashSet[pointer]()
  reg.ev[typehash].incl cb

proc disconnect(reg: Reg, ty: typedesc, cb: pointer) =
  const typehash = hash($ty.type)
  if not reg.ev.hasKey(typehash): return
  reg.ev[typehash].excl cb

proc trigger(reg: Reg, ev: auto) =
  const typehash = hash($ev.type)
  if not reg.ev.hasKey(typehash): return
  for pcb in reg.ev[typehash]:
    type pp = proc (ev: ev.type) {.nimcall.}
    cast[pp](pcb)(ev)

when isMainModule:
  var reg = Reg()
  
  type
    MyEvent = object
      hihi: string
      hoho: int
    SomeOtherEvent = object
      ents: seq[int]
  
  var obj = "From bound obj: asdads" # A bound obj
  
  proc cbMyEvent(ev: MyEvent)  =
    echo "my event was triggered", ev.hihi, ev.hoho
    echo obj
  
  proc cbMyEvent2(ev: MyEvent) =
    echo "my event was triggered TWO", ev.hihi, ev.hoho
    echo obj
  
  reg.connect(MyEvent, cbMyEvent) # these are good
  reg.connect(MyEvent, cbMyEvent2) # these are good
  reg.disconnect(MyEvent, cbMyEvent2) # these are good
  
  # Now the issue: i can bind an invalid combination of event and callback:
  # thanks to my use of pointers...
  reg.connect(SomeOtherEvent, cbMyEvent) # this should be disallowed! Breaks on runtime
  
  var myev = MyEvent()
  myev.hihi = "hihi"
  myev.hoho = 1337
  trigger(reg, myev)
  
  myev.hihi = "HAHAHAH" # change the event a little
  reg.trigger(myev)
  
  var sev = SomeOtherEvent()
  sev.ents = @[1,2,3]
  trigger(reg, sev) # Breaks on runtime...

the code works, but the problem i face is that i loose type safety. So even if a callback cannot operate on the given type. The use of pointers let me still connect it. Then it crashes on runtime.

I think the solution is, to make the connect proc typesafe. But i cannot get it working. What i tried:

proc connect2[E](reg: Reg, cb: proc (ev: E)) =
  const typehash = hash($E)
  if not reg.ev.hasKey(typehash):
    reg.ev[typehash] = initHashSet[pointer]()
  reg.ev[typehash].incl unsafeAddr cb

## Then later:
var sev = SomeOtherEvent()
sev.ents = @[1,2,3]
trigger(reg, sev)
# SIGSEGV: Illegal storage access. (Attempt to read from nil?)

what it think the issue is that i store the pointer to the attribute. But i'm not quite sure.

Any idea how to do this properly?

enthus1ast (orginal) [2022-01-08T12:42:53+01:00] view original

one thing that seems to work is to use system.rawProc

proc connect2[E](reg: Reg, cb: proc (ev: E)) =
  const typehash = hash($E)
  if not reg.ev.hasKey(typehash):
    reg.ev[typehash] = initHashSet[pointer]()
  reg.ev[typehash].incl rawProc cb

planetis (orginal) [2022-01-08T17:49:37+01:00] view original

How about not using events with ECS. I mean there is nothing stopping you, but event driven programing is more related to OOP. At least in my experience I used simple empty components as 'Tags' that notify the next system of changes. They can be added and removed very efficiently. Of course you might end up in a situation where there are system ordering issues and a dirty solution other people recommend is to run some system twice in a tick. At least it's really simple...

enthus1ast (orginal) [2022-01-08T19:07:14+01:00] view original

Imho events in an ecs are an essential building block for real decoupling. Without them the systems are intertwined.

Eg.:

An entity receives damage. Without an event system the inflictDamage proc must call eg the playDamageSound or the createBlood proc etc. Therefore the inflictDamage proc must know all the others.

With an event system the inflictDamage proc just emits evInflictDamage event. Nothing else must be known. Other functionality connects from the outside and are therefore easy to add or remove without changing other unrelated parts of the code.

kobi (orginal) [2022-01-09T08:16:00+01:00] view original

A good event system would be great to have, though in the past I found it hard to debug. Recently I've been looking into Dart (for Flutter actually), and perhaps what you need is state management. A nice approach to decouple code I saw there was the BloC pattern. here is an example to see how it looks: https://www.youtube.com/watch?v=nQMfaQeCL6M Though maybe I am misreading what you need.

coffeepot (orginal) [2022-01-09T21:45:05+01:00] view original

@planetis

At least in my experience I used simple empty components as 'Tags' that notify the next system of changes. They can be added and removed very efficiently.

In general, I agree with this. In many cases using components as "events" works better than explicit events because they can be easily inspected and modified, and their work can be split over multiple systems in a clearly defined order. Supporting ordering, inspection, and work splitting tends to be more complex/less transparent with a callback/queue style event system (YMMV, of course).

Having said that, I found myself implementing events in my ECS despite it being a huge complexity burden (in my case, events are immediate and allow mutating entities during system iteration). I rarely use them because, as you say, tags are often more ergonomic.

Why bother, then? Events allow defining behaviour for state transitions, and for that, they are invaluable. They round out design options. It's one of those things that you never need... until you do!

Events are particularly useful for context sensitive initialisations:

KillAfter.onInit:
  # Event records when this component is added.
  curComponent.startTime = cpuTime()

makeSystem "updateShrinkAway", [ShrinkAway, KillAfter]:
  # System to normalise the time remaining.
  let curTime = cpuTime()
  all:
    item.shrinkAway.normTime = 1.0 - (
      (curTime - item.killAfter.startTime) / item.killAfter.duration
    )

makeSystem "shrinkAwayModel", [ShrinkAway, Model]:
  # System to shrink and fade out 3D models.
  # A similar system handles [ShrinkAway, FontText].
  added:
    # Event invoked when ShrinkAway and Model first exist together.
    item.shrinkAway.startScale = item.model.scale[0]
    item.shrinkAway.startCol = item.model.col
  all:
    item.model.scale = vec3(item.shrinkAway.startScale * item.shrinkAway.normTime)
    item.model.col = item.shrinkAway.startCol * item.shrinkAway.normTime

@enthus1ast

It may be worth noting that iteration order in HashSet is undefined if event order matters for you. There's OrderedSet though, if you don't need other set operations.

An entity receives damage. Without an event system the inflictDamage proc must call eg the playDamageSound or the createBlood proc etc. Therefore the inflictDamage proc must know all the others.
With an event system the inflictDamage proc just emits evInflictDamage event. Nothing else must be known. Other functionality connects from the outside and are therefore easy to add or remove without changing other unrelated parts of the code.

Potentially, this can be data-driven with just components:

makeSystem "inflictDamage", [Damage, Health]:
  all:
    item.health.amount -= item.damage.amount
    if item.health.amount <= 0.0:
      entity.add Killed() # Tag this entity for deletion after sound and FX.
      entity.add PlaySound(name: "boom", volume: 1.0)
    elif item.health.amount < someThreshold:
      # Blood particles.
      entity.add Particles(amount: 20, col: vec3(1.0, 0.0, 0.0), speed: -0.03 .. 0.03)
      entity.add PlaySound(name: "bleed", volume: 1.0)
    else:
      # Minor damage particles.
      entity.add Particles(amount: 5, col: item.model.col, speed: -0.02 .. 0.02)
      entity.add PlaySound(name: "scuff", volume: 0.5)

makeSystem "soundEvents", [PlaySound]:
  all:
    # Play sounds.
  sys.remove PlaySound  # Remove PlaySound for entities in this system.

makeSystem "particleFx", [Pos, Particles]:
  all:
    for i in 0 ..< item.particles.amount:
      discard newEntityWith(
        item.pos,
        Vel(x: rand item.particles.speed, y: rand item.particles.speed),
        Graphic(...)
      )
  sys.remove Particles

makeSystem "kill", [Killed]:
  # Delete all entities with Killed.
  sys.clear

Anyway, I hope this isn't derailing too much! Just wanted to add some other perspectives on why events are useful in an ECS context, and how components themselves can act as events.

jyapayne (orginal) [2022-01-10T04:22:23+01:00] view original

@enthus1ast, why not something using Nim's generics? Is there a reason you wanted to use pointers?

For example, you could do something like this (made a little better with macros):

import tables
import macros

type
  Callback[T] = proc (arg: T)
  Signal[T] = ref object
    connections: Table[string, Callback[T]]

proc connect[T](signal: Signal[T], callback: Callback, name: string) =
  signal.connections[name] = callback

proc disconnect[T](signal: Signal[T], name: string) =
  signal.connections.del name

proc trigger[T](signal: Signal[T], val: T) =
  for connection in signal.connections.values:
    connection(val)

macro connect[T](signal: Signal[T], callback: untyped): untyped =
  let name = callback.toStrLit
  result = quote do:
    connect(`signal`, `callback`, `name`)

macro disconnect[T](signal: Signal[T], callback: untyped): untyped =
  let name = callback.toStrLit
  result = quote do:
    disconnect(`signal`, `name`)

when isMainModule:
  
  type
    MyEvent = object
      hihi: string
      hoho: int
    SomeOtherEvent = object
      ents: seq[int]
  
  var reg = Signal[MyEvent]()
  
  var obj = "From bound obj: asdads" # A bound obj
  
  proc cbMyEvent(ev: MyEvent)  =
    echo "my event was triggered", ev.hihi, ev.hoho
    echo obj
  
  proc cbMyOtherEvent(ev: SomeOtherEvent)  =
    echo "my event was triggered", ev.ents
    echo obj
  
  proc cbMyEvent2(ev: MyEvent) =
    echo "my event was triggered TWO", ev.hihi, ev.hoho
    echo obj
  
  reg.connect(cbMyEvent) # these are good
  reg.connect(cbMyEvent2) # these are good
  reg.disconnect(cbMyEvent2) # these are good
  
  #reg.connect(cbMyOtherEvent, "myotherevent") # this now fails at compile time
  
  var myev = MyEvent()
  myev.hihi = "hihi"
  myev.hoho = 1337
  trigger(reg, myev)
  
  myev.hihi = "HAHAHAH" # change the event a little
  reg.trigger(myev)
  
  var sev = SomeOtherEvent()
  sev.ents = @[1,2,3]
  #trigger(reg, sev) # Breaks at compile time

Is that flexible enough?

enthus1ast (orginal) [2022-01-10T08:39:57+01:00] view original

This can unfortunately just bind one event type. That's the reason i used pointers in the first place. But i like the macros.

sky_khan (orginal) [2022-01-10T13:47:49+01:00] view original

Not sure if this is the best way but this seems working:

import tables, typetraits

type
  EventBase = object of RootObj
  
  EventProc = proc(ev:EventBase) {.nimcall.}
  EventCallback[T:EventBase] = proc(ev:T) {.nimcall.}
  
  EventReg = object
    events : Table[string,seq[EventProc]]

proc connect[T:EventBase](reg: var EventReg,callback: EventCallback[T]) =
  let n = name(T)
  echo "Connecting ",n
  if not reg.events.hasKey(n):
    reg.events[n] = @[]
  let cb = cast[EventProc](callback)
  reg.events[n].add(cb)

proc disconnect[T:EventBase](reg: var EventReg,callback: EventCallback[T]) =
  let n = name(T)
  echo "Disconnecting ",n
  if not reg.events.hasKey(n): return
  let cb = cast[EventProc](callback)
  let ndx = reg.events[n].find(cb)
  if not ndx<0:
    reg.events[n].delete(ndx)

proc trigger[T:EventBase](reg:EventReg,ev:T) =
  let n = name(ev.typedesc)
  echo n & " is triggered"
  if not reg.events.hasKey(n): return
  for cb in reg.events[n]:
    cb(ev)

when isMainModule:
  var reg = EventReg()
  
  type
    KeyPressedEvent = object of EventBase
      keycode: int
    
    InflictDamageEvent = object of EventBase
      damage : float
    
    SomeOtherEvent = object of EventBase
      id : int
      name : string
  
  proc handleKeyPress1(ev:KeyPressedEvent)=
    echo "Keypress1: ", ev.keycode
  
  proc handleKeyPress2(ev:KeyPressedEvent)=
    echo "Keypress2: ", ev.keycode
  
  proc handleInflictDamage(ev:InflictDamageEvent) =
    echo "You have ", ev.damage, " less health"
  
  proc handleSomeOtherEvent(ev:SomeOtherEvent) =
    echo "Some other ", ev.id, " ", ev.name
  
  var kev = KeyPressedEvent()
  kev.keycode = 1234
  
  var idev = InflictDamageEvent()
  idev.damage = 10
  
  var sev = SomeOtherEvent()
  sev.id = 4321
  sev.name = "Hello"
  
  connect[KeyPressedEvent](reg,handleKeyPress1)
  connect[KeyPressedEvent](reg,handleKeyPress2)
  connect[InflictDamageEvent](reg,handleInflictDamage)
  connect[SomeOtherEvent](reg,handleSomeOtherEvent)
  disconnect[KeyPressedEvent](reg,handleKeyPress1)
  
  #connect[KeyPressedEvent](reg,handleInflictDamage) #-> type mismatch
  
  trigger[KeyPressedEvent](reg,kev)
  trigger[InflictDamageEvent](reg,idev)
  trigger[SomeOtherEvent](reg,sev)

DeletedUser (orginal) [2022-01-10T13:57:35+01:00] view original

The solution here is to generate an object from a list of event types which generates a seq of procs field for each type, then generate a proc for dispatching each event type based on its field. It requires the registry type to know about all the event types beforehand. Otherwise there is no way for it to be type safe.

jyapayne (orginal) [2022-01-11T05:13:01+01:00] view original

Ah, I misunderstood what you were trying to do. @Hlaaftana is correct, in order for type safety to remain, Nim needs to know what types you want to include in the event registry at compile time. I took a very brief stab at a rough version using macros (that can certainly be improved), and here's what I came up with:

import tables, strutils
import macros


macro registerEvents(body: untyped): untyped =
  
  template createTriggerProc() =
    proc disconnect(reg: Registry, name: string) =
      for evObj in reg[].fields:
        evObj.connections.del name
  
  template createEventProcs(fieldName, tyName) =
    proc connect(reg: Registry, callback: proc (arg: tyName), name: string) =
      reg.fieldName.connections[name] = callback
    
    proc trigger(reg: Registry, val: tyName) =
      for connection in reg.fieldName.connections.values:
        connection(val)
  
  result = newNimNode(nnkStmtList)
  var regRecList = nnkRecList.newNimNode
  var registryTypeDef = nnkTypeDef.newTree(
      ident("Registry"),
      newEmptyNode(),
      nnkRefTy.newTree(
        nnkObjectTy.newTree(
          newEmptyNode(),
          newEmptyNode(),
          regRecList
        )
      )
    )
  
  var typeSection = nnkTypeSection.newNimNode
  result.add typeSection
  
  for node in body:
    let signalName = ident($node.toStrLit & "Signal")
    let tyName = ident($node.toStrLit)
    let lowerTyName = ident(($node.toStrLit).toLowerAscii)
    
    result.add getAst(createEventProcs(lowerTyName, tyName))
    
    regRecList.add nnkIdentDefs.newTree(
      lowerTyName,
      signalName,
      newEmptyNode()
    )
    
    let procTy = nnkProcTy.newTree(
      nnkFormalParams.newTree(
        newEmptyNode(),
        nnkIdentDefs.newTree(
          ident("arg"),
          tyName,
          newEmptyNode()
        )
      ),
      newEmptyNode()
    )
    
    typeSection.add nnkTypeDef.newTree(
      signalName,
      newEmptyNode(),
        nnkObjectTy.newTree(
          newEmptyNode(),
          newEmptyNode(),
          nnkRecList.newTree(
            nnkIdentDefs.newTree(
              ident("connections"),
              nnkBracketExpr.newTree(
                ident("Table"),
                ident("string"),
                procTy
              ),
              newEmptyNode()
            )
          )
        )
    )
  
  typeSection.add registryTypeDef
  
  result.add getAst(createTriggerProc())


type
  MyEvent = object
    hihi: string
    hoho: int
  SomeOtherEvent = object
    ents: seq[int]

# This is all you need to do for each type you want to be in the registry, but can only be called once
# Can be improved to take a name for the Registry type
registerEvents:
  MyEvent
  # MyEvent2

# These macros currently need to be defined where ever you call `registerEvents`, otherwise you
# must specify the names of the events manually
macro connect(reg: Registry, callback: untyped): untyped =
  let name = callback.toStrLit
  let regName = reg.toStrLit
  
  template doConnect(reg, regName, callback, name) =
    when compiles(connect(reg, callback, name)):
      connect(reg, callback, name)
    else:
      {.error: "'" & name & "' event not found in registry '" & regName & "'".}
  return getAst(doConnect(reg, regName, callback, name))

macro disconnect(reg: Registry, callback: untyped): untyped =
  let name = callback.toStrLit
  template doDisconnect(reg, name) =
    disconnect(reg, name)
  return getAst(doDisconnect(reg, name))



var reg = Registry()

var obj = "From bound obj: asdads" # A bound obj

proc cbMyEvent(ev: MyEvent)  =
  echo "my event was triggered", ev.hihi, ev.hoho
  echo obj

proc cbMyOtherEvent(ev: SomeOtherEvent)  =
  echo "my event was triggered", ev.ents
  echo obj

proc cbMyEvent2(ev: MyEvent) =
  echo "my event was triggered TWO", ev.hihi, ev.hoho
  echo obj

reg.connect(cbMyEvent) # these are good
reg.connect(cbMyEvent2) # these are good
reg.disconnect(cbMyEvent2) # these are good

reg.connect(cbMyOtherEvent) # this now fails at compile time with "Error: 'cbMyOtherEvent' event not found in registry 'reg'"

var myev = MyEvent()
myev.hihi = "hihi"
myev.hoho = 1337
trigger(reg, myev)

myev.hihi = "HAHAHAH" # change the event a little
reg.trigger(myev)

var sev = SomeOtherEvent()
sev.ents = @[1,2,3]
trigger(reg, sev) # Breaks at compile time

The registerEvents macro invocation above creates this equivalent code (which can be printed using echo result.toStrLit):

type
  MyEventSignal = object
    connections: Table[string, proc (arg: MyEvent)]
  
  Registry = ref object
    myevent: MyEventSignal

proc connect(reg: Registry; callback: proc (arg: MyEvent); name: string) =
  reg.myevent.connections[name] = callback

proc trigger(reg: Registry; val: MyEvent) =
  for connection in reg.myevent.connections.values:
    connection(val)

proc disconnect(reg: Registry; name: string) =
  for evObj in reg[].fields:
    evObj.connections.del name

jyapayne (orginal) [2022-01-11T05:14:53+01:00] view original

@sky_khan, your solution seems to work too, although I don't completely trust Nim's inheritance due to past issues. Maybe those are ironed out now though

enthus1ast (orginal) [2022-01-11T16:41:23+01:00] view original

Not only this, but its also quite slow I've benchmarked @sky_khan s system (I like it) but it's twice as slow than mine with pointers

sky_khan (orginal) [2022-01-11T17:33:04+01:00] view original

IDK, You may encounter bugs if you make a class hierarchy from EventBase as @jyapayne said but I dont think it would be 2x slower if you change it as yours. I mean, I was lazy and used type name as keys instead of hash and used sequence as container for events. That must be the reason, I guess

enthus1ast (orginal) [2022-01-11T18:32:45+01:00] view original

I've patched it a little, to use const hashes and copy the api I previously had. Can post it later if you like, even if its slower I still consider useing it.

sky_khan (orginal) [2022-01-11T18:42:44+01:00] view original

No need for me but maybe others can give a fix / suggestion

geekrelief (orginal) [2022-01-25T18:17:05+01:00] view original

Just following the conversation. it'd be nice if you post what you finally end up with.

demotomohiro (orginal) [2022-01-26T02:17:34+01:00] view original

Here is simplified code reproduces same runtime error. This code cause SIGSEGV because it reads an address of cb, and stores it to gpointer. Using unsafeAddr without understanding pointer and address is as dangerous as going to grocery without wearing mask in pandemic.

import sugar

var gpointer: pointer

proc setGPointer(cb: proc ()) =
  # stores an address of cb
  gpointer = unsafeAddr cb
  dump gpointer.repr
  dump cb.repr

proc callGPointer() =
  # SIGSEGV: Illegal storage access. (Attempt to read from nil?)
  cast[proc () {.nimcall.}](gpointer)()

proc myCallback() =
  echo "Calling myCallback"

setGPointer(myCallback)
callGPointer()

Using rawProc might looks like solving the error, but you might see strange output when closure is used. Using rawProc or closure without knowing about procedural type or calling conventions is as dangerous as going to grocery without wrecking bar in zombie apocalypse.

This section explains about procedural type and calling conventions: https://nim-lang.org/docs/manual.html#types-procedural-type

var gpointer: pointer

proc setGPointer(cb: proc ()) =
  # What is the default calling convention for a procedural type Nim manual says?
  gpointer = rawProc cb

proc callGPointer() =
  cast[proc () {.nimcall.}](gpointer)()

proc myCallback() =
  echo "Calling myCallback"

setGPointer(myCallback)
# Works
callGPointer()

proc getClosure(x: int): proc() =
  # Is this print x correctly?
  (proc () = echo "Calling from closure in getClosure:", x)

setGPointer(getClosure(123))
callGPointer()

Output:

Calling myCallback
Calling from closure in getClosure:10

setGPointer in follwing code works only with nimcall calling convention and it doesn't use rawProc. Calling it with closure is compile time error.

var gpointer: pointer

proc setGPointer(cb: proc () {.nimcall.}) =
  gpointer = cb

proc callGPointer() =
  cast[proc () {.nimcall.}](gpointer)()

proc myCallback() =
  echo "Calling myCallback"

setGPointer(myCallback)
# Works
callGPointer()

# Following code is compile time error
proc getClosure(x: int): proc() =
  (proc () = echo "Calling from closure in getClosure:", x)

setGPointer(getClosure(123))
callGPointer()

You cannot store a closure type by casting it to single pointer because it have a pointer to proc and a pointer to environment. I don't know how to store different types of closures to a seq safely without using object variants.

Mirror of forum.nim-lang.org

8784 :: simple event system