I'm running into this problem in every project of mine, and I simply can't find a good solution, although the problem seems very basic.
Let's say I write a dummy lib.nim like this:
# The lib provides a definition of an abstract base type
# without any actual instantiations...
type
Element* = ref object of RootObj
# Example for a field that should be an implementation
# detail, used only internally in `lib`. It should
# neither be public nor should there be getter/setter
# so that we can rely on the fact that it does not
# change after construction.
id: string
proc run*(elements: openarray[Element]) =
for element in elements:
echo "Touching element: " & element.id
The library is intended to be instantiated on user side, for example:
import lib
type
ElementA = ref object of Element
ElementB = ref object of Element
# But how to construct them?
proc newElementA(): ElementA =
ElementA(id: "A")
proc newElementB(): ElementB =
ElementB(id: "B")
let elements = @[
newElementA().Element,
newElementB(),
]
run(elements)
This doesn't compile because the user has no access to id at all, even in the constructor. Usually I provide a constructing proc in the scope of the type, e.g.:
# in lib.nim
proc newElement*(id: string): Element =
Element(id: id)
Now users can construct an Element, but I'm not sure if it helps constructing subtypes. This crashes at runtime with an invalid conversion (kind of expected):
proc newElementA(): ElementA =
result = newElement("A").ElementA
I also don't want to add setter procs/templates to lib.nim because than the field gets fully exposed.
Am I missing some basic template/macro tricks to solve this basic problem? I have spend hours going back and forth making fields private and public again or moving code around because I can't seem to get the visibility right.
I tried it once this way:
https://forum.nim-lang.org/t/4226#26316
After fixing the wrong var parameter type I think it was working as expected, but indeed we should have some lecture notes from the bright devs in a Nim textbook, together with the system.proccall usage for methods.
@Stefan_Salewski: That what I meant with mutable setters ;). I'm trying to avoid them so that user can cannot arbitrarily reset the field.
I finally had an idea that I can at least abstract away the mutation of the field:
# in lib.nim
template newElement*(idString: string, body: typed): untyped =
let element = body
element.id = idString
element
On user side construction now works like this:
type
ElementA = ref object of Element
a: int
ElementB = ref object of Element
b: int
proc newElementA(): ElementA =
newElement("A", ElementA(a: 1))
proc newElementB(): ElementB =
newElement("B", ElementB(b: 2))
Not sure if there is something nicer, but this doesn't look too bad to me...
In this direction, to make it work, you have to do a cast:
proc newElementA(): ElementA =
cast[ElementA](newElement("A"))
Another way consists to create a proc initElement in lib:
proc initElement*(elem: Element, id: string) =
elem.id = id
then to define newElementA this way:
proc newElementA(): ElementA =
new result
result.initElement("A")
@lscrd: The initElement is also a setter despite its name ;). You cannot stop users from calling element.initElement("different id") and mutating the field at any point in time, which is what I'm trying to avoid.
In fact that is also true for my template constructor idea via newElement("different id", existingElement). If anything the template makes it a bit less obvious. Maybe what I really need is a macro taking a typedesc/generic and a list of constructor args that are forwarded to the subtype construction. Then the macro can really only be used for constructing new elements with injected id modifications.
I struggled with this as well several times in several libraries before giving up and assume that people don't hack my internal data structure. Here was my latest trial:
type
PublicKey* = object
Fraw_key: array[64, byte]
PrivateKey* = object
Fraw_key: array[32, byte]
# Hide the private fields and generate accessors.
# This is needed to:
# - Be able to not store the public_key in PrivateKey in the future and replace it by
# an on-the-fly computation
# - Signature: have different data representation
template genAccessors(name: untyped, fieldType, objType: typedesc): untyped =
# Access
proc name*(obj: objType): fieldType {.noSideEffect, inline, noInit.} =
obj.`F name`
# Assignement
proc `name=`*(obj: var objType, value: fieldType) {.noSideEffect, inline.} =
obj.`F name` = value
# Mutable
proc `name`*(obj: var objType): var fieldType {.noSideEffect, inline.} =
obj.`F name`
genAccessors(raw_key, array[64, byte], PublicKey)
genAccessors(raw_key, array[32, byte], PrivateKey)
The accessor are only visible for the internal library proc but not exported outside of the library. However returning a var type was quite tricky.
Also var result array generates a null pointer dereference and can't be used at all: https://github.com/nim-lang/Nim/issues/8053.
Now for ref types with visible fields there is also the issue of shallow vs deep immutability. You cannot have deep immutability for ref types at the moment. See https://github.com/nim-lang/Nim/issues/8370 for the latest discussion.
Yes, you are right :-(. I forgot your requirement.
I fear there is no simple solution in this case, as you need to create the object with the right type (to get the right size) and, at this moment, you don’t have access to the private field. In fact, I’m sure that my first solution with a cast doesn’t work if extra fields exists in ElementA (it compiles, but will cause problems at runtime).
And I’m not sure that using a macro will change anything: you need to allocate in the client module to get the right size and to allocate in the lib module to have access to the private field. This is incompatible.
But there are ways to check that initElement is not called several times. For instance, you can add a private boolean field initialized in Element which is false by default. Then initElement becomes:
proc initElement(elem: Element, id: string) =
if elem.initialized:
# already initialized: raise some exception
…
elem.id = id
elem.initialized = true
Or, if you require that id cannot be the empty string, you no longer need the initialized field:
proc initElement(elem: Element, id: string) =
if id.len == 0:
# id must not be empty: raise some exception.
…
if elem.id.len != 0:
# already initialized: raise some other exception.
…
elem.id = id
I agree that these are not very elegant solutions. Maybe using a macro to initialize could solve the problem in a more elegant way. I can’t say for sure as I have not use macros for now.
The accessor are only visible for the internal library proc but not exported outside of the library.
In this case I cannot rely on hiding the fields via re-exporting, because the library is by design supposed to be extended on user side. Think of an UI component library, where users should be able to define their own components.
@lscrd Good points about the runtime checks, but yes compile time security would be preferred.
I have now found two solutions that seem to satisfy the requirements:
# on lib side
template privateInitializer(element: typed, idString: string): untyped =
element.id = idString
element
macro newElement*(T: typedesc, id: string, args: varargs[untyped]): untyped =
# convert varargs into an object constructor call of T
let constructor = newTree(nnkObjConstr)
constructor.add(T.getTypeInst[1])
for arg in args:
expectKind(arg, nnkExprEqExpr)
constructor.add(newColonExpr(arg[0], arg[1]))
# apply post construction initialization of parent fields
result = newCall(bindSym"privateInitializer", constructor, id)
# on user side
newElement(ElementA, id="A", a=1)
# on lib side
macro verifyObjectConstructor(x: typed): untyped =
if x.kind != nnkObjConstr:
error($x.repr[0..^1] & " is not an object constructor")
template newElement*(idString: string, constructor: typed): untyped =
verifyObjectConstructor(constructor)
let element = constructor
element.id = idString
element
# on user side
newElement("A", ElementA(a: 1))
# abusing the template as an arbitrary setter is now a compile time error:
let el = ElementA(a: 1)
newElement("A", el)
@Araq: What is your opinion on this? I'm wondering if it would make sense to differentiate visibility for construction vs field access. As far as I can see a lot of visibility problems would be solved if there was an intermediate between fully-hidden and fully-exposed which is "exposed in object construction". Would it for instance make sense to have
type
Element* = ref object of RootObj
id {.initializable.}: string
so that public access to id is prevented in general, but id can be passed into the constructor including subtype constructors?
Alternatively, one could use a generic constructor:
# lib.nim
import typetraits
type
Element* = ref object of RootObj
id: string
proc newElement*(id: string, T: typedesc[Element] = Element): T =
result.new
result.id = id
echo "Instantiated an object of type ", T.name, ", size: ", result[].sizeOf
proc printId*(e: Element) =
echo e.id
# user.nim
import lib
type
ElementA = ref object of Element
n: int
proc newElementA(id: string, n: int): ElementA =
result = newElement(id, ElementA)
result.n = n
let
e = newElement("base")
ea = newElementA("A", 5)
e.printId
ea.printId
echo ea.n
@Rania_d Note that the issue you linked is only a problem in languages that don't allow named parameters. It's not a problem in Nim because it has named parameters, so no need for the builder pattern.
Nim's problem is rather that it doesn't allow to combine immutable objects with inheritance out of the box.
@gemath Good idea, so it comes down to either setting the additional fields of the parent object vs the fields of the derived object.
@Araq: Great to hear that there is some activity regarding write tracking, my most anticipated Nim feature ;).
I think this is an independent problem though, that rather relates to object construction / initialization, not really related to immutability (my title was misleading). I'm not sure if it is worthy of a RFC, but I tried to sum it up more clearly here:
https://github.com/nim-lang/RFCs/issues/145
It could very well be that I'm missing something ;).