I've now played around with templates and found that with all the pragmas (ordinary/immediate/dirty/inject/gensym) they are impossible to understand. I more or less stumbled around to find something that worked, for benchmark, maybe I'm just too new to nim and its "obvious" to others, but I think its overly complicated.
I'm guessing it can be simplified, but maybe not as its already been discussed to death. Anyway, before putting to any effort into a proposal I thought I'd check to see if anyone else agrees and what the communities current thoughts are.
Go ahead and make a proposal. I'm curious how you would simplify them. In the meantime, here is yet another attempt to explain them:
dirty: "everything is resolved in the instantiation context"
gensym: "generate a fresh temporary variable here for every instantiation to resemble function call semantics"
inject: "the instantiation scope sees this symbol"
immediate: "don't resolve types and expand this thing eagerly"
normal: "resolve types and participate in overloading resolution"
winksaville: I think its overly complicated.
Those difficulties are pretty much inherent to any sufficiently expressive macro system, though. You can make it simpler, but only at the cost of making it also less powerful. You could arguably tweak things a bit (there may be more intuitive names for some of the features, for example), but I don't see how you can make the system appreciably simpler without also weakening it.
Personally, I think the best way to make templates more approachable would be to work on the documentation instead.
I'll also add that more most of your day-to-day programming, you don't really need templates (or at least no more than the defaults, which is essentially an inlining mechanism).
I agree the current documentation is lacking and before making a proposal we need to understand the definitions (thanks Araq) and what problems templates are solving and what knobs (pragmas) are currently needed to solve them.
Do you want to discuss things here or should I create an issue or maybe use a Google doc, what would be best?
I agree with Jehan that some complexity is inherent. But I think it could be made more approachable by renaming some of these (specially 'gensym' which doesn't seem clear to me - maybe it should be 'local' or 'hidden' or something if I understand correctly).
Here is my understanding - they are easier to grasp in pairs:
@vbtt, pairing them up makes for nice summary.
But things are more complex and I think the extracted paragraphs highlight that fact:
Per default templates are hygienic: Local identifiers declared
in a template cannot be accessed in the instantiation context.
Whether a symbol that is declared in a template is exposed
to the instantiation scope is controlled by the inject and
gensym pragmas: gensym'ed symbols are not exposed but
inject'ed are.
The default for symbols of entity type, var, let and const
is gensym and for proc, iterator, converter, template, macro
is inject. However, if the name of the entity is passed as
a template parameter, it is an inject'ed symbol.
The inject and gensym pragmas are second class annotations;
they have no semantics outside of a template definition and
cannot be abstracted over.
To get rid of hygiene in templates, one can use the dirty
pragma for a template. inject and gensym have no effect
in dirty templates.
In paragraph 1 it states clearly "Local identifiers declared in a template cannot be accessed in the instantiation context". Then in paragraph 3 it says "type, var, let and const is gensym" (i.e. not instantiated) and then goes on to say "proc, iterator, converter, template, macro is inject" (i.e. is instantiated). So paragraph 3 contradicts paragraph 1, at least that's my understanding. So we end up with 4 concepts we have to juggle.
I'd like to suggest the that we require the template author to explicitly define what is "instantiated" and what is "hidden" there would be no default. But, rather than doing it individually I suggest we have pragmas/keywords such as "local" and "visible" (or maybe "instantiated/hidden"). This way no rule has to be remembered. The author easily gets the "dirty" effect by placing everything in the "visible" section. I would also allow multiple occurrences of "local" "visible" within a template.
By doing the above we eliminate 4 concepts; "hygenic", "dirty", "inject" and "gensym" plus remembering a special rule and replace them with two conecpts "local" and "visible".
To ease the transition I'd suggest we define another name for these new templates, at <http://www.powerthesaurus.org/template> there are 56 synonyms for template, I kinda like "stencil", but there are 55 others plus whatever some good wordsmith might think up :)
So this is my first suggestion, my second is to get rid of immediate and only have ordinary, but I need someone to explain why we need immediate.
Paragraph one merely explains what hygienic means and then a more elaborate explanation follows.
The point of most of these rules is to have sensible defaults. This system evolved from a much simpler system which was "tuned" against many real world examples.
But, rather than doing it individually I suggest we have pragmas/keywords such as "local" and "visible" (or maybe "instantiated/hidden"). This way no rule has to be remembered.
These pragmas/keywords are not any better than what we have now and having no default simply sucks! gensym should obviously be the default for local variables.
By doing the above we eliminate 4 concepts; "hygenic", "dirty", "inject" and "gensym" plus remembering a special rule and replace them with two conecpts "local" and "visible".
The model is simpler to grasp when you learn templates by reading their specification (which I argue is a bad way to learn anything), but it's harder to use!
Having "local/gensym" as the default is reasonable.
By what measure are you saying my proposal is harder to use?
By what measure are you saying my proposal is harder to use?
Well again, the current system evolved from a simpler system which was "tuned" against many real world examples. So I can definitely imagine what a simpler system looks like.
For instance, "ok, make gensym the default for everything!". Brilliant! One rule less to learn!
But then this:
template liftCmpFromField(typ: typedesc, field: expr): stmt =
proc `<` (x, y: typ): bool = x.field < y.field
proc `<=` (x, y: typ): bool = x.field <= y.field
proc `==` (x, y: typ): bool = x.field == y.fieldNeeds to become this:
template liftCmpFromField(typ: typedesc, field: expr): stmt =
proc `<` * (x, y: typ): bool {.inject.} = x.field < y.field
proc `<=` * (x, y: typ): bool {.inject.} = x.field <= y.field
proc `==` * (x, y: typ): bool {.inject.} = x.field == y.fieldWhich makes it harder to use.
With my suggestion it would be:
template liftCmpFromField(typ: typedesc, field: expr): stmt =
{.visible.}
proc `<` * (x, y: typ): bool = x.field < y.field
proc `<=` * (x, y: typ): bool = x.field <= y.field
proc `==` * (x, y: typ): bool = x.field == y.field
or
template liftCmpFromField(typ: typedesc, field: expr): stmt =
visible:
proc `<` * (x, y: typ): bool = x.field < y.field
proc `<=` * (x, y: typ): bool = x.field <= y.field
proc `==` * (x, y: typ): bool = x.field == y.field
I'm not sure but this seems to do what you want:
- template semiImmediate(x: int; body: untyped) =
- let foo = x+4 body
- semiImmediate(56):
- echo "abc"
Note that untyped is simply a new alias for expr which reflects reality much better.
My thought for not using immediate was so we could overload the template, but probably because expr is 'untyped' we still can't overload it, at least this doesn't work:
template suite(param: int, suiteBody: expr): stmt =
block:
proc myEcho(s: string) =
echo s
suiteBody
template suite(suiteBody: expr): stmt =
suite(0, expr)
suite:
myEcho("suiteBody of suite no <param>")
suite 1:
myEcho("suiteBody of suite 1")
suite 2:
myEcho("suiteBody of suite 2")
$ nim c -r --hints:off t4.nim
t4.nim(11, 2) Error: undeclared identifier: 'myEcho'
For an example of its usefulness I think my "suite" example is useful, its the model I used in benchmark and I'd like to have at least two benchmark suite implementations:
suite(name: string, warmupSeconds: float, suiteBody: untyped)
suite(name: string, suiteBody: untyped) =
suite(name, DEFAULT_WARMUP_SECONDS, suiteBody)
In any case, I'm glad to here its possible, which may mean it will be implemented someday :)
Workaround:
suite(name: string, warmupSeconds: untyped, suiteBody: untyped)
suite(name: string, suiteBody: untyped) =
suite(name, DEFAULT_WARMUP_SECONDS, suiteBody)
Note how warmupSeconds and 'suiteBody' agree on 'untyped' so that the 2nd argument can always remain untyped for overloading resolution.
Thanks, as you see below its working on my example! Its not with benchmark, but it's more complex so I'll need to look deeper at what I'm doing wrong.
templa"te suite(param: untyped, suiteBody: untyped): stmt =
block:
var iparam: int = param
proc myEcho(s: string) =
echo s
myEcho("param=" & $iparam)
suiteBody
template suite(suiteBody: untyped): stmt =
suite(0, suiteBody)
suite:
myEcho("suiteBody of suite 0 no <param>")
suite 1:
myEcho("suiteBody of suite 1")
suite 2:
myEcho("suiteBody of suite 2")
param=0
suiteBody of suite 0 no <param>
param=1
suiteBody of suite 1
param=2
suiteBody of suite 2
The problem with benchmark is that I was using the unittest module and it also defines a template unittest.suite and I needed to "fully qualify" the template benchmark.suite.
So initially I had the following and this generated a "wrong number of arguments" it looks like 'suite "loop tests":' was resolved to 'ut.suite "loop tests":'
import unittest as ut
import benchmark
ut.suite "benchmark tests":
ut.test "benchmark test and with and without setup/teardown":
var suiteCount = 0
check(suiteCount == 0)
suite "loop tests":
....
test "loop 10", 10, tsArray: ## <<< This generated "Error: wrong number of arguments"
....
import unittest as ut
import benchmark as bm
ut.suite "benchmark tests":
ut.test "benchmark test and with and without setup/teardown":
var suiteCount = 0
check(suiteCount == 0)
bm.suite "loop tests":
....