I am looking to write a syntax tree scanner. I have a list of procedures- call them foo, bar and fuz- and I want to know if any of them are called in a code block.
The point of the exercise is to have a template that behaves a bit differently depending on the block passed to it. The template is good enough for everything except the scanning, for which it would call a macro containsFooBarOrFuz that returns bool.
So as far as I came was to have a typed macro (so templates containing calls to foo, bar and fuz are expanded before scanning) that will recursively scan each tree-node and check the type and name of each regular node, returning a true node if there is a match and false in the end if nothing did
I was going to go ahead and start writing it when it occurred to me some of you may have already done something similar I possibly could adapt.
Can you help? Thanks!
macro containsCall(procNames: static seq[string], body: typed): bool =
let procNames = procNames
proc search(node: NimNode): bool =
case node.kind
of nnkCall, nnkCommand: $node[0] in procNames or node[1..^1].any(search)
of AtomicNodes: false
else: node.toSeq.any(search)
newLit(search(body))
I have considered writing a more in-depth thing about macros and the logic surrounding them lately. One of the things I've learnt by working with macros over the years is that trying to parse real Nim code is never a good idea. In general the only time you should parse the AST is when writing a DSL. For introspection uses it is a better idea to use multiple passes, creating invokeable templates, and/or using compile time variables to track things. These are by far the more robust solution.
Unfortunately they require a bit more knowledge of the language, and more specific knowledge of the problem in order to create a good solution. If you can share more details about your problem I'm sure we could find a nice robust solution.
And there are of course cases where you just end up parsing the code. I'm just saying that in general it's better to avoid it if you want your macro to compose with other things.
Thanks for your feedback! Robust solution is why I'm here!
My limdb-module allows creating a transaction object, reading or writing to it with a table-like interface, and then either rolling it back or committing to it. For a read-only transaction, by convention you roll back. The transaction handles write order and provides a consistent view of the data.
import limdb
let db = initDatabase("pathToDb")
let t = db.initTransaction()
t["foo"] = "bar"
t["fuz"] = "buz"
t.commit # committing
let t2 = db.initTransaction()
let foo = t["foo"]
let fuz = t["fuz"]
t.reset # by convention, a read-only transaction is rolled back
I want to create a withTransaction template that will automate the transaction details to make it safer. LMDB just locks up if you forget to reset or commit a transaction.
import limdb
let db = initDatabase("pathToDb")
db.withTransaction(t):
t["foo"] = "bar"
t["fuz"] = "buz"
# implicit t.reset
var foo
var bar
db.withTransaction(t):
foo = t["foo"]
bar = t["buz"]
# implicit t.reset
So since I already have initTransaction, reset and commit ready, it would appear having a `withTransaction template is rather straight-forward with the exception of whether picking between to commit or reset, which is where the containsCall proc comes in. containsCall would not need be univeral, relying on for example the first parameter being a Transaction object is fine. So good point on asking for details...
I'd personally suggest just making a
db.withCommitTrans(it):
it["foo"] = "bleh"
it["bar"] = "blaaah"
db.withResetTrans(it):
foo = it["foo"]
bar = it["bar"]
Perhaps even destructors with two distinct types make sense? I do not know anything about DBs aside from how to spell it.
That simple!!! Impossible. How can it be good if it's not complicated?
😉
Actually that may be a way better idea. More control...
But I'd still like to learn how to do the automatic version if it's not out-of-this-world complicated, it has its charm as well.
That'll do it.
I was thinking though it might be too inflexible to have to decide on whether to commit or not before the transaction is complete. Then I noticed it static analysis might not be enough to make that distinction- it may be better to simply have a 'written' flag in the transaction object that []= et al set, and commit if it's true.
On the other hand, there's no real harm in committing a transaction without writing. Not sure what the best design tradeoffs are here for most cases:
I like about (2) that it's the simplest API and doesn't have any overhead but I'm not sure if I should just go for (1) and call it a day, since it appears to be more readable. (3) might be a bit more error-prone than (1) or (2), maybe if you need that much flexibility you should commit/reset manually.
It's pretty bad DSL design. Here is a better one:
Thanks!
Could you be a tiny bit more specific about what's bad about it, aside of naming?
Thank you!
In @Araq's defense, you can always raise an exception to trigger a reset.
And withTransaction only responds to compile-time logic. If you have a runtime-conditional write, you always get a commit, as proposed.
You know, maybe I ought to go with withTransaction not because it's better or worse, but to be a contrarian.
You know, maybe I ought to go with withTransaction not because it's better or worse, but to be a contrarian.
The world needs more contrarians but I can almost guarantee the AST inspection that you perform is neither complete (what if the call to commit is in some helper proc you don't traverse) nor correct (what if it's a commit call that happens to be overloaded for some unrelated type).
I stopped parsing the AST and I'm using tag-tracing now, so if someone missed anything, it was probably you!
In all seriousness, this ought to cover all the cases. If you call the C-API in the indented code block you had it coming.
I also just switched to generic types in the generics branch to natively support all basic types and alliw you to roll your own toBlob/fromBlob do everything can go through the tagged API.
I stopped parsing the AST and I'm using tag-tracing now, so if someone missed anything, it was probably you!
Ok, fair enough.
Aw shucks, discovered an issue
db.withTransaction:
t["fuz"] = "buz"
if false:
t.commit
# does not add anything
# transaction not ended, program blocks
Statically ensuring a tagged proc is called in all code paths seems more trouble than it's worth.
@Araq's approach really is simple to implement (like old Unix tools 😉) Maybe with different naming? "hopefully obvious" might work as well as "hopefully familiar"...
# (a)
db.readonly:
t["fuz"] = "buz"
# always adds reset
db.writable:
t["fuz"] = "buz"
# always adds commit
Or perhaps supershort, familiar and autochoose (no manual commit/reset)
# (b)
with db:
t["foo"] = bar
Or the same by default with more control
# (c)
type Mode = enum[auto, readonly, writable]
with db:
t["foo"] = bar
# commits
with db readonly:
t["foo"] = bar
# resets
The agony.
The enum solution indicates that you can do things like:
proc flexibleButPointless(m: Mode) =
with db, m:
t["foo"] = bar
which never come up in reality, but the macro needs to protect against. It's only a static keyword away though so it's not too bad.
Thanks for the warning- and for your extended disussion for that matter!
Darnit, I think I'm going for (b).