It seems like JSON in Nim is really slow.
It's a bit unfair to compare with Node.JS, as in Node JSON parsed by heavily optimised C code. Yet, I think the numbers should be more or less comparable, not orders of magnitude slower. The scripts below are 30 times slower with -d:release and 160 times slower without it.
Nim timing
> nim r play.nim
Time taken: 5.182976999999999 sec
> nim -d:release r play.nim
Time taken: 0.845689 sec
Node.JS timing
deno run tmp/play.ts
Time taken: 0.032 sec
Nim code
import times, os, std/json, std/jsonutils
type OptionContract* = ref object
id*: string
right*: string
expiration*: string
strike_raw*: float
premium_raw*: float
data_type*: string
type OptionChain* = object
contracts*: seq[OptionContract]
proc stub_data(): OptionChain =
result = OptionChain()
for _ in 1..6000:
result.contracts.add OptionContract(
id: "AMZN CALL 2021-03-19 1460.0 USD",
right: "call",
expiration: "2021-03-19",
strike_raw: 1460.0,
premium_raw: 1676.03,
data_type: "some type"
)
let json_str = stub_data().to_json.pretty
let time = cpuTime()
for i in 1..5:
discard json_str.parse_json.json_to(OptionChain)
echo "Time taken: ", cpuTime() - time, " sec"
Node.JS code
interface OptionContract {
id: string
right: string
expiration: string
strike_raw: number
premium_raw: number
data_type: string
}
interface OptionChain {
contracts: OptionContract[]
}
function stub_data(): OptionChain {
let contracts: OptionContract[] = []
for (let i = 0; i < 6000; i++) contracts.push({
id: "AMZN CALL 2021-03-19 1460.0 USD",
right: "call",
expiration: "2021-03-19",
strike_raw: 1460.0,
premium_raw: 1676.03,
data_type: "some type"
})
return { contracts }
}
let json_str = JSON.stringify(stub_data(), null, 2)
let time = Date.now()
for (let i = 0; i < 5; i++) {
JSON.parse(json_str)
// In JS the JSON parsed by native C-code, so let's
// do some work in JS-land
.contracts.map((c: any) => ({...c, id: "abc " + c.id }))
}
console.log(`Time taken: ${(Date.now() - time) / 1000} sec`)
Nim's std lib parsing is kind of slow. That is why I made jsony ( https://github.com/treeform/jsony )
import times, os, std/json, jsony
type OptionContract* = ref object
id*: string
right*: string
expiration*: string
strike_raw*: float
premium_raw*: float
data_type*: string
type OptionChain* = object
contracts*: seq[OptionContract]
proc stub_data(): OptionChain =
result = OptionChain()
for _ in 1..6000:
result.contracts.add OptionContract(
id: "AMZN CALL 2021-03-19 1460.0 USD",
right: "call",
expiration: "2021-03-19",
strike_raw: 1460.0,
premium_raw: 1676.03,
data_type: "some type"
)
let json_str = stub_data().toJson()
let time = cpuTime()
for i in 1..5:
discard json_str.fromJson(OptionChain)
echo "Time taken: ", cpuTime() - time, " sec"
Time taken: 0.042 sec
Time taken: 0.033 sec
Hmm typescript is still faster. Nim's standard json parsers first creates intermediate json nodes then turns them into nim types, causing each object and string to be double allocated. This is slow. Jsony reads json directly and allocates objects once kind of like TypeScript does.
Jsony could be faster at this benchmark if I would have written a custom float parser. This is some thing I have not done yet. Current float parser allocates some garbage that slows it down.
The benchmarking could use work, discard there might allow the compiler to just throw a way the results and whole computations. Also I recommend running benchmark multiple times to get min, average and standard divination. I have written a library for this called benchy ( https://github.com/treeform/benchy ). Using benchy:
timeIt "parsing":
for i in 1 .. 5:
keep json_str.fromJson(OptionChain)
name ............................... min time avg time std dv runs
parsing ........................... 40.118 ms 42.083 ms ±1.599 x118
I think I could get Nim to beat TypeScript if I work on the float parser.
You can use https://github.com/treeform/jsony instead of std/json.
Modified code:
import benchy, jsony
import times, os, std/json, std/jsonutils
type OptionContract* = ref object
id*: string
right*: string
expiration*: string
strike_raw*: float
premium_raw*: float
data_type*: string
type OptionChain* = object
contracts*: seq[OptionContract]
proc stub_data(): OptionChain =
result = OptionChain()
for _ in 1..6000:
result.contracts.add OptionContract(
id: "AMZN CALL 2021-03-19 1460.0 USD",
right: "call",
expiration: "2021-03-19",
strike_raw: 1460.0,
premium_raw: 1676.03,
data_type: "some type"
)
let json_str = jsonutils.to_json(stub_data()).pretty
timeIt "std/json":
for i in 1..5:
discard json_str.parse_json.json_to(OptionChain)
timeIt "jsony":
for i in 1..5:
discard json_str.fromJson(OptionChain)
Result:
name ............................... min time avg time std dv runs
std/json ......................... 464.545 ms 481.940 ms ±6.026 x11
jsony ............................. 37.933 ms 38.547 ms ±0.749 x129
Thanks Ward! Its pretty cool you chose my library as first thing.
I have committed some changes to the float parser but I still can't beat typescript, but I did gain about 3.5ms of speed.
name ............................... min time avg time std dv runs
parsing ........................... 36.510 ms 37.093 ms ±0.913 x134
Though if this is a one shot script, you might consider disabling GC with --gc:none then it can beat type script:
nim c -r -d:danger --gc:none .\tests\bench_options.nim
name ............................... min time avg time std dv runs
parsing ........................... 23.764 ms 27.883 ms ±9.115 x17
JS is GC and JIT is very fast for a dynamic language. My hope that nim's GC will improve, but there are still tricks to "cheat" and come out ahead. This is what irked me about javascript while making my browser based game, once you hit the JS performance wall you can't optimize any more. With nim there are things todo!
That is why I made jsony
Yes, thanks. I knew about it, but it was more or less tolerable so far, so I was continue to use std/json till now.
Nim's standard json parsers first creates intermediate json nodes then turns them into nim types, causing each object and string to be double allocated.
Hmm, maybe not. I specifically modified JS example and allocated the extra copy, this extra-allocation in JS is not even visible in benchmark. So, extra allocation should be cheap, there's something else that slow down Nim json.
I got a better result even than --gc:none by adding following code and compiling with nim r -d:danger --opt:speed --gc:arc.
proc parseHook*(s: string, i: var int, v: var string) {.inline.} =
v = newStringOfCap(64)
jsony.parseHook(s, i, v)
timeIt "jsony":
for i in 1..5:
discard json_str.fromJson(OptionChain)
Another optimization can be done by using view to rewrite parseSymbol.
benchmarks should use -d:danger, not -d:release, or at least show both
I disagree about -d:danger. Safety is one of Nim core features. Benchmarking with key feature disabled would be wrong, as it would be benchmark for something totally artificiall that never going to be used in practice.
slower compared to what nim version?
1.4.2 without jsonutils -> 1.4.6 with jsonutils (became slower)
JSON.parse(json_str) already gets you a js object on which to operate, whereas in json or jsonutils you have an intermediate JsonNode representation:
I modified JS to have intermediate representation, it doesn't slow it down
for (let i = 0; i < 5; i++) {
JSON.parse(json_str)
// In JS the JSON parsed by native C-code, so let's
// do some work in JS-land
.contracts.map((c: any) => ({
id: "" + c.id,
right: "" + c.right,
expiration: "" + c.expiration,
strike_raw: 0 + c.strike_raw,
premium_raw: 0 + c.premium_raw,
data_type: "" + c.data_type
}))
}
There is a bunch of issues related to JSON performance: #12833 #3809 #12152 and nim-lang/RFCs#188
Perhaps it's worth linking them in https://nim-lang.org/docs/json.html and showing a big warning on the top of the page.
@alexeypetrushin can you explain? > 1.4.2 without jsonutils -> 1.4.6 with jsonutils (became slower)
your snippet uses jsonutils, so how can you compare apples to apples with a version that doesn't have it? please post a version that works with some prior version and is slower in devel than that prior version, otherwise it's impossible to validate/investigate
your snippet uses jsonutils, so how can you compare apples to apples with a version that doesn't have it?
I used 1.4.2 without jsonutils, and updated recently to 1.4.6 and also updated json code to use the new jsonutils in all my json related code. Here's the diff. The difference between two versions on my machine is 0.17 and 0.83 sec with -d:release option.
Have you tried sam?
I'm considering swithing to jsony, but thanks will check it out too :)
@alexeypetrushin I've investigated this and fixed the performance problem, see https://github.com/nim-lang/Nim/pull/18183
now jsonutils.jsonTo doesn't have any overhead compared to json.to (the PR actually makes a 20x speed improvement compared to before PR)