Heyho, I've been playing around with implementing reactivex (or rather expanding an existing lib to more fully implement reactivex), which is a lib to implement reactive programming, aka observables, subjects etc. . This allows you to make values depend on one another via a push instead of a pull mechanism. There I stumbled into a problem.
Strictly speaking, the concepts of reactive x create a graph of 3 types of nodes:
So more conceptually speaking I want to implement a graph that will not contain cycles, but can strictly speaking in totality have an arbitrary amount of generic types.
That naturally runs into a problem with the type system, because the implementation I can think of would roughly look like this:
type Subject[T] = ref object # Source node
observer: Observer[T]
type Observable[T] = ref object # Source node
observer: Observer[T]
value: T
type ParentKind = enum
pkObservable, pkSubject, pkOperator
type Operator[T, S] = ref object # Intermediate/Operator node
case kind: ParentKind:
of pkObservable:
obsParent: Observable[T]
of pkSubject:
subjParent: Subject[T]
of pkOperator:
opParent: Operator[R, T] # <-- Wait, so `Operator` needs the third parameter R, but now this might be Q, R, T, S if your parent operator now has Q, R, T... etc.
The problem is the second you want to chain nodes (aka your parent is another Operator-Node) then you also need the types of the parent. So suddenly instead of generic types T and S, you also need R. But then you also may have a parent-node with that has a parent, node. In that scenario your OperatorNode type suddenly needs also Q, and so on and so forth.
Is there some kind of way to solve this in a fashion that doesn't create limitations?
If RXJS is anything to go by, then this seems like it might be impossible to solve in an unlimited fashion and I should just define a dozen or so Operator types with various amounts of generic types. Because they apparently gave up looking at their type-declarations for their pipe function
pipe(): Observable<T>;
pipe<A>(op1: OperatorFunction<T, A>): Observable<A>;
pipe<A, B>(op1: OperatorFunction<T, A>, op2: OperatorFunction<A, B>): Observable<B>;
pipe<A, B, C>(op1: OperatorFunction<T, A>, op2: OperatorFunction<A, B>, op3: OperatorFunction<B, C>): Observable<C>;
pipe<A, B, C, D>(
op1: OperatorFunction<T, A>,
op2: OperatorFunction<A, B>,
op3: OperatorFunction<B, C>,
op4: OperatorFunction<C, D>
): Observable<D>;
pipe<A, B, C, D, E>(
op1: OperatorFunction<T, A>,
op2: OperatorFunction<A, B>,
op3: OperatorFunction<B, C>,
op4: OperatorFunction<C, D>,
op5: OperatorFunction<D, E>
): Observable<E>;
.... they play this game up to I
Related:
https://github.com/ReactiveX/rxjs/issues/4221
https://github.com/ReactiveX/rxjs/issues/5139
Looking at docs for other Rx implementations for statically typed languages, e.g. RxJava and RxSwift, pipe doesn't seem to be in the picture, but Transformer and compose are a thing in RxJava.
RxJS started out implemented in plain JavaScript, gained TypeScript typings leading on the way to RxJS v5, and was re/implemented in TypeScript for v5. I could be totally wrong, but I'm guessing some of the design of the v5+ impl takes into consideration JS-only devs that don't care about TS, devs that use it exclusively, and others in the middle.
Off the cuff I can't quote see how that eliminates my problem which I currently mostly see in Operator storing a Ref to a parent-instance which may have an unknown amount of generic params. With turning callbacks into closures I can eliminate some of the types, but that not all of them as I'd need.
The key here is that Observers may get called N times, every time a new value gets pushed through the pipeline. So I'm not executing the closure just once or multiple times with the same value, I execute it multiple times with different values each time.
So I could only imagine this in this kind of construct:
type Observer[T] = ref object
value: T
subscription: proc() {.closure.}
proc newObserver[T](subscription: proc(value: T)): Observer[T] =
let obs = Observer[T](value: default(T))
proc subscriptionClosure() {.closure.} =
subscription(obs.value)
obs.subscription = subscriptionClosure
return obs
let obs = newObserver(proc(x: int) = echo "Value is: ", x)
obs.subscription() # value is: 0
obs.value = 4
obs.subscription() # value is: 4
Which gets rid of the type in the callback I need to call, but now I need an external value to manipulate for the closure I need to call, so I still have a T.
I think I just stumbled onto something thanks to Elegantbeef in the discord. I can use closure, but more for fetching values from "parent"-Observables (to erase their types) than the observers.
type Observable[SOURCE] = ref object
value: SOURCE
getValue: proc(): SOURCE {.closure.}
observer: seq[proc(value: SOURCE)]
proc newObservable[SOURCE](value: SOURCE): Observable[SOURCE] =
Observable[SOURCE](
value: value,
getValue: proc(): SOURCE = value
)
type OperatorObservable[SOURCE, RESULT] = ref object
getValue: proc(): RESULT {.closure.}
transformer: proc(value: SOURCE): RESULT
observer: seq[proc(value: RESULT)]
proc newOperatorObservable[A, B, C](
parent: OperatorObservable[A, B],
transformer: proc(value: B): C
): OperatorObservable[B, C] =
proc getValueClosure(): C =
echo "Getting value from Operator Obs"
let parentValue: B = parent.getValue()
return transformer(parentValue)
return OperatorObservable[B, C](
getValue: getValueClosure,
transformer: transformer,
observer: @[]
)
proc newOperatorObservable[A, B](
parent: Observable[A],
transformer: proc(value: A): B
): OperatorObservable[A, B] =
proc getValueClosure(): B =
echo "Getting value from Source Obs"
let parentValue: A = parent.getValue()
return transformer(parentValue)
return OperatorObservable[A, B](
getValue: getValueClosure,
transformer: transformer,
observer: @[]
)
let obs = newObservable[int](5)
let doubleObs = newOperatorObservable(
obs,
proc(x: int): int =
echo "Double: ", x * 2
return x * 2
)
let quadrupleObs = newOperatorObservable(
doubleObs,
proc(x: int): int =
echo "Quadruple: ", x * 2
return x * 2
)
let toStrObs = newOperatorObservable(
quadrupleObs,
proc(x: int): string = "I am now a string! "
)
echo quadrupleObs.getValue()
echo toStrObs.getValue()
I'll see whether this will drive me into a corner somewhere, but I'm feeling optimistic about this approach.