I use custom JSON generation code, it's controlled by exporting to_json_hook proc.
The issue is that it's not imported properly, behaving in a weird way.
The code below should print [{"text":"Some text"}] but instead it prints [{"isUnquoted":false,"kind":5,"fields":{"text":{"isUnquoted":false,"kind":4,"str":"Some text"}}}]. Caused by the to_json_hook not being exported properly.
There are 2 solutions - see the 1) note. And, the really weird the 2) note. All three files are needed to showcase this strange behavior.
Why it is so, and is there a way to avoid this surprising behavior?
json_helper.nim
import std/json, std/jsonutils
export json, jsonutils
proc to_json_hook*[T: tuple](o: T): JsonNode =
result = new_JObject()
for k, v in o.field_pairs: result[k] = v.to_json
proc to_json_hook*(n: JsonNode): JsonNode = n
plot.nim
import std/json, std/jsonutils
# 1 Uncomment this and it wold work. It shouldn't
# be needed. As this proc will be exported from
# `json_helpers` in `main`.
# proc to_json_hook*(n: JsonNode): JsonNode = n
type Page* = ref object
page*: seq[JsonNode]
proc text*(page: var Page, text: string): void =
page.page.add (text: text).to_json
# 2 Or, even stranger, comment out this method and it
# would work too
proc image*(page: var Page, data: JsonNode): void =
page.page.add (image: data).to_json
main.nim
import plot, json_helpers
var page = Page()
page.text "Some text"
echo page.page.to_json
@alexeypetrushin, is this surprising behavior? I think it should be expected since you are not importing json_helpers.nim inside plot.nim, and that's where the conversion inside the text proc happens. So there's no way for plot.nim to know about your json hook.
You can modify the import of plot.nim to this to get it to work:
import json_helpers # since you already export the other to modules
So there's no way for plot.nim to know about your json hook.
If I use different JSON settings, how can I publish the plot.nim library? Other people have different hooks.
So there's no way for plot.nim to know about your json hook.
Actually, no it doesn't make sense. The JSON conversion stuff defined in std/json I define json hook in other modules. And yet std/json somehow knows about those hooks. So why this case is different?
Please have a look at https://nim-lang.org/docs/manual.html#generics-symbol-lookup-in-generics
Thank you!
As far as I understand the example above should work then? Because to_json (defined in std/jsonutils) is a generic that uses overloaded open symbol to_json_hook (defined in my json_helpers.nim lib).
So it doesn't matter if my to_json_hook proc is visible in plot.nim or not. Or do I miss something?
P.S.
The only thing is that the text proc defined in plot.nim (which calls generic to_json) is not generic, but even if i change it do generic, the example still doesn't work.
Your example isn't generic.
Replace this
proc text*(page: var Page, text: string): void =
page.page.add (text: text).to_json
proc image*(page: var Page, data: JsonNode): void =
page.page.add (image: data).to_json
proc text*[T:Page](page: var T, text: string) =
page.page.add (text: text).to_json
proc image*[T:Page](page: var T, data: JsonNode) =
page.page.add (image: data).to_json
Or you could replace
import plot, json_helpers
import json_helpers
include plot
However the order is important in this case and plot will only compile when included in another file that already imports json_helpers.
Your example isn't generic.
Thanks, I tried that, it doesn't work. It's actually even stranger. Note in all the examples proc text is used and proc image isn't used at all, yet it will affect how the code behaves.
Case 1 remove unused proc image and it would work, even with non-generic proc text. Seems like a bug, as according to manual this case should not work.
import std/json, std/jsonutils
type Page* = ref object
page*: seq[JsonNode]
proc text*(page: var Page, text: string) =
page.page.add (text: text).to_json
They are not bugs. You just don't understand how generics work versus non generics.
Araq was mistaken at first and corrected himself when he said,
Please have a look at https://nim-lang.org/docs/manual.html#generics-symbol-lookup-in-generics
Unfortunately the first example isn't quite complete enough.
type
Index = distinct int
proc `==` (a, b: Index): bool {.borrow.}
var a = (0, 0.Index)
var b = (0, 0.Index)
echo a == b # works!
when false:
proc `==`[T:tuple](a,b : T) : bool #from system
# approximately how it's defined
for value,value2 in fields(a,b):
if value != value2:
return false
return true
type
Index = distinct int #not generic
#defined after `==` from system
proc `==` (a, b: Index): bool {.borrow.} #not generic
var a = (0, 0.Index)
var b = (0, 0.Index)
#calls proc `==`[T:tuple](a,b : T) : bool with (int,Index) as T
#worsk even though proc is defined before borrow proc
#because proc from system is generic
echo a == b
start here for further information on generics https://nim-lang.org/docs/manual.html#generics
and read until https://nim-lang.org/docs/manual.html#generics-symbol-lookup-in-generics
I finally distilled it into a short example, play with 2min video demo explaining where's the problem.
Problem: comment out the code marked in comments, which is completely unrelated, and it would produce different results.
type JsonNode = tuple
content: string
proc to_json_default*(s: string): JsonNode =
(content: "default " & s)
proc to_json_default*(o: tuple): JsonNode =
# Case 1 replace this proc body with discard
for k, v in o.field_pairs: discard v.to_json
proc to_json*[T](v: T): JsonNode =
when compiles(v.to_json_hook): v.to_json_hook else: v.to_json_default
# Case 2 remove this unrelated proc
proc something_totally_unrelated*(text: string): void =
discard (text: text).to_json
proc to_json_hook*(s: string): JsonNode = (content: "hook " & s)
echo "v".to_json.content
P.S.
There are couple more strange things if you make the example a bit more complicated and introduce generics, see Case 1, 2, 3 comments.
This is expected behavior. Your unrelated proc calls to_json with the exact same types as
echo "v".to_json
if you change
proc to_json*[T](v: T): JsonNode =
when compiles(v.to_json_hook): v.to_json_hook else: v.to_json_default
template to_json*[T](v: T): JsonNode =
when compiles(v.to_json_hook): v.to_json_hook else: v.to_json_default
A new version of a generic proc is created for each combination of types its instantiated with. Since you already created to_json with a string argument your later invocation of it uses the previously created proc.
Turning the proc into a template fixes the problem because calling a template is similar to copy pasting the body of the template wherever it's called.
Since you already called to_json with a string argument your later invocation of it uses the previously created proc.
Thanks, I see now, thanks. I couldn't imagine it could work this way :)
if you change proc to template your example will work as expected.
Indeed, it works. Although I think it make situation worse, now it's even more complicated. It's not just 3 cases - proc vs generics & templates & its order, but 4 cases - proc vs generics vs templates vs its order.
I think it this approach is wrong and broken. You remove some unrelated code or change order of some procs, and your program behaves differently. The good thing is that such cases are relativelly rare.