nimforum mirror - Flow Based Programming in Nim

drifter (orginal) [2016-10-01T18:43:02+02:00] view original

Hi there, I've been reading up on a programming model referred to as Flow-Based Programming (FBP).

http://www.jpaulmorrison.com/fbp/ https://en.wikipedia.org/wiki/Flow-based_programming

I'm wondering if any of you have implemented something like this in Nim?

Central to FBP is the concept that data is to be seen as the primary "thing" in an application, This "thing" moves from one process to another, undergoing a constant series of transformations.

Processes receive and send packets of information between each other.

The sole purpose of a process is to transform the data it receives into another form, before sending it off to another process.

A few other highlights:

A process has IN and OUT channels (kind of like a socket's receive and send)

Processes are connected together (ex: connect("ProcessA.OUT", "ProcessB.IN") to create a network of interconnected processes

Data (Information Packets) is received by the receiving process(es) in the order in which they were sent by the sending process

Once they have finished transforming the data they have received, a receiving process sits iddle until new data is sent to it.

I could see model working well for making the jump from process diagrams into code implementation, almost having a 1:1 relationship. Well, especially in my line of work as a IT Biz Analyst where we do a lot of process diagrams. Mapping a process diagram to a process using this approach would seem trivial.

The only thing is I'm not quite sure how to implement this in Nim (well, in any langauge in general).

The author of "Flow Based Programming", Paul Morrison, pointed me to a couple of implementations of this approach:

http://www.jpaulmorrison.com/fbp/jsyntax.htm

This approach has been implemented in Java, in C++, Javascript. He tells me implementation of this approach depends on the language being used supporting threads (which I know Nim does). But I have to admit I'm not strong enough a programmer to figure it out on my own.

So if anyone is interested, and have a few ideas or pointers, I hope you don't mind chiming in!

Regards,

runvnc (orginal) [2016-10-02T03:25:32+02:00] view original

Flow or dataflow programming is very interesting and powerful.

I think some similar concepts are streams and channels which both have modules for Nim: http://nim-lang.org/docs/channels.html http://nim-lang.org/docs/streams.html

You can also think of flow as a more general abstraction for function composition (with flexible input and output streams usually). If you want to go that route, look through the Nim manual and documentation about string templating, and then metaprogramming with macros, the the way the parser works. You might also take advantage of source code filters http://nim-lang.org/docs/filters.html for generating diagram markup like XML, or even Nim code from in-memory graphs of workflow models.

Here someone has a stream implementation in Nim with pipes https://github.com/zielmicha/reactor.nim/blob/master/reactor/async/stream.nim

Here is a workflow engine (related to flow programming and business process management) in C++ https://github.com/kluthen/libworkflow which could be either integrated with a Nim program or have the code converted.

Here is a list of workflow engines https://github.com/pditommaso/awesome-pipeline. Some of those written in python might not be hard to convert to Nim or interact with.

There is a nimqml library. QML could be used to make flow diagrams/editors https://github.com/jehrichs/Diaqml

Here is a web-based example of NoFlo which is JavaScript based (BTW Nim can also compile to JavaScript in addition to C and C++): http://app.flowhub.io/#example/04847249319d2326fa92 Also for JavaScript I believe is https://bpmn.io/toolkit/bpmn-js/ https://npms.io/search?q=BPMN

Channels are an intrinsic part of the Go programming language: https://gobyexample.com/channels

If you are not already a programmer then you may have a long journey to get to an implementation. https://zapier.com/ is along the same lines but oriented towards business users. Also see https://bpmn.io/ and https://camunda.org/

EDIT: sorry, there is another way that flow-based programming is incorporated into Nim. See the threadpool module code which declares something like a FlowVarBase. The flow-style execution is needed because Nim supports futures with keywords like await, which will cause execution to pause until input for that part of the control flow is available, while allowing other input/output to continue at the same time.

Anyway it is an interesting general question so thanks for posting it. Looks like I just spent an hour googling flow-related stuff. Which I guess I have to admit is part of a fun and easy Saturday for me. Perhaps collecting so many links was a bit self-indulgent.

drifter (orginal) [2016-10-06T00:52:49+02:00] view original

Runvnc,this was a quite a detailed answer :) Ihave to thank you for taking the time for chiming in and providing all of these links.

Seems I have quite a bit of reading to do.

Thank very much,

Mirror of forum.nim-lang.org

2561 :: Flow Based Programming in Nim