Hey guys, I've been using the dotOperators feature since I started my game engine project 2 years ago. I use it to interface with C++ libraries, and for the most part it works amazingly. Being able to implicitly generate Nim bindings to a C++ library as you use it in your code is sick, IMO. I also use it to implement GLSL/HLSL-like field swizzling for my math stuff.
As useful as it is, I have encountered some limitations with it. dotOperators has .=, but it doesn't have something like .+=, or .*=, or .<arbitrary operator>. It would be cool to be able to do something like vector4.yzx *= 2, and it doesn't seem like extending dotOperators to support this scales well.
I don't know anything about language design, but a thought occurred to me that might address this issue. Right now, Nim has computed properties using the property= syntax (would also be cool to have property+=, etc.). What if there was a special macro/template that could generate procs like this, without explicitly naming them? Something like:
type Obj = object
macro `=call`(obj: Obj, op: untyped, args: varargs[untyped]): untyped =
echo op.repr
let instance: Obj
instance.test = 1 # prints 'test='
instance.test += 1 # prints 'test+='
discard instance.test # prints `test`
`test*=`(instance, 1) # prints `test*=`; probably not feasible, have to mention anyway :)
You might be onto something. Working through a bunch of examples of it working and also acting poorly will help test whether it's a good idea.
Generalizations that reduce the number of rules in a language are typically a very welcome thing, fewer concepts more leverage. One way to do that is the RFC process.
Hope that helps.
In order to improve some existing design you need to enumerate the problematic aspects of the current design and then see how a different design avoids these problems. I mind dot operators quite a bit because in ordinary Nim code obj.f means that an operation named f exists, so at least this "name lookup" cannot fail at runtime. This is a criticism directed at using the "dot syntax" so if your alternative keeps the dot syntax it doesn't address this problem at all.
However, there are other problems with dot operators, for example, implementation complexity that could be addressed with your design.
Thanks @saem and @Araq. I've been posting utterly bad RFCs in the RFC repo, and wanted to get people's input before I embarrass myself yet again. Your replies really helped.
Since I got your attention, please let me pick your brains so I can write some semblance of a good RFC.
If I understood Araq correctly, dotOperators is bad because it allows code that cannot be proven statically to be valid at runtime (field access in this case). Does the Nim compiler have to be able to catch cases like this, or is it enough if the underlying backend (e.g. the C++ compiler) catches this when compiling the generated C/C++ code?
The reason I ask this is because one of my usecases for dotOperators is to generate valid C++ code without having to define every type and method that is declared in the underlying C++ frameworks I am interacting with. As a result, there are cases where Nim might generate C++ code that doesn't compile, and I have to debug the generated code itself to figure things out.
Below is an example of the usecase I'm talking about. I've added some implementation details to help visualize the mechanics of it:
## interop.nim
# used to represent any C++ type; more specific C++ types can inherit from this
type CType* {.inheritable importcpp:"auto".} = object
# declares a DSL to declare C++ symbols
template namespace*(name: untyped) =
# ...
# specifies the path to the header file for the C++ symbols defined in its body
macro header(path: static[string], body: untyped) =
#...
# declares a C++ class/enum, and optionally its fields and methods
macro class(decl: untyped, body: untyped = void) =
#...
macro invocation(code: string, sym: auto, name: untyped, retT: type, args: varargs[untyped]): auto =
let decl = genAst(code, name, retT):
proc name(self: CType): retT {.importcpp:code varargs.}
let call = genAst(name, sym):
name(sym)
for arg in args:
call.add(arg)
result = newStmtList(decl, call)
# used to generate field access for C++ type without explicitly declaring the fields in Nim
template `.`*(self: CType, field: untyped): auto =
invocation("#.$1", self, field, var CType)
## declare.nim
header "path/to/include.hpp":
class CppType:
var field1: int
let field2: float
proc fn(): string
## use.nim
let obj = make CppType()
# type-checked by Nim:
obj.field1 = 1
# obj.field2 = 2 # error
# Nim compiles this without problems, but VCC might not:
obj.undeclaredField.undeclaredNestedField = "C++ might not compile"
Being able to partially declare C++ types and use them immediately like above is extremely useful to me, but I think this capability necessarily means that compile errors may need to come after Nim has generated the C++ code. Like I said in my OP, my goal is to extend this -- and the other usecase I talked about -- hopefully without complicating the compiler even further. Do you think this is worth basing an RFC on, or is this utter nonsense?
I know I'm looking at this purely with the C/C++ backends in mind, and there are still the JS and the future LLVM ones. Would love to get a hint about how those backends might be affected by this idea as well.
In general if I see a GCC error I consider that a bug in the Nim compiler, not in my code. (Unles I'm lying to the compiler that imaginary functions exist, then it's my fault) but relying on a GCC error for a Nim feature would be very poor imho.
Ignoring the distraction of interop, your use case might point to a way to mitigate @Araqs concern. If a dot operator is sugar for the creation of a proc plus it's invocation, (nonanonymous lambda? notorious lambda?) wouldn't that satisfy the requirement that "obj.f exist at runtime
My "gut reaction" is that extending the dot operator in such a way would generate more problems than it would solve from the point of view of the programmer's understanding of what is happening.
Perhaps use a different symbol sequence than a dot? That way the programmer has a "here be dragons" warning.
An example:
obj+>undeclaredField+>undeclaredNestedField = "C++ might not compile"
@JohnAD I think I agree with your gut reaction. I have tried a few times to replace the ., .=, and .() a few times already with -> for exactly this reason, but I couldn't find a way to replicate all of my usecases with it. I guess I could bite the bullet and just use something like cppObject.tryCall(CppMethod, args) -- which definitely screams "here be dragons" -- but after using something like dotOperators this lessens "programmer happiness" on my part.
@shirleyquirk yes relying on the underlying backend to find and fix typos is definitely one of the weaknesses of my interop design. I do find it worth the trouble though given that the porting aspect of interoping with C++ is taken out of the equation and I go straight to prototyping. I have tried other solutions like generating the C/C++ interfaces with tools like nimterop, but it simply does not work with the libraries I use.
Thanks guys for your input. Perhaps a better direction to go with this is to completely eliminate the need for dot operator overloading and instead allow similar capabilities but using other operators like ->?
but I couldn't find a way to replicate all of my usecases with it.
Please continue to use the dot operators until a better solution emerges.