Mirror of forum.nim-lang.org

8537 :: TechEmpower Benchmarks

• v3ss0n (orginal) [2021-10-25T06:50:16+02:00] view original

  Nim httpbeast was 8th , 2 benchmarks ago : https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=json

  Now it is 20th https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=json

  I am curious what changed the performance?

---

• miran (orginal) [2021-10-25T07:41:56+02:00] view original

  I am curious what changed the performance?

  Other frameworks got faster?

---

• v3ss0n (orginal) [2021-10-25T13:43:43+02:00] view original

  benchmark 20:

  avg : 0.3ms sd : 0.3ms max: 17.8 ms

  benchmark 18:

  avg : 0.3ms sd : 0.1ms max: 12.5 ms

  so from those benchmarks , nim got slower in benchmark 20 : which is done at 2021-02-08 benchmark 18 was at : 2019-07-09

---

• dom96 (orginal) [2021-10-25T15:46:01+02:00] view original

  Definitely an interesting question to explore. For what it's worth, taking for example tokio-minihttp in the JSON benchmark, it was #6 in Round 18 and is #68 in Round 20.

  I think it's a combination of a lot of things: new languages/frameworks added, new variants of the same (already fast) languages/frameworks being added, some slowing down of Nim and/or the libraries. What it comes down to is as usual time commitment: if somebody spends time identifying the regressions/bottlenecks they can help us get to the top.

---

• PMunch (orginal) [2021-10-25T16:08:33+02:00] view original

  Got a nice speedup just from passing -flto -march=native. Also tried to update various versions (Ubuntu, httpbeast, Nim) without any huge benefits, and --gc:arc was worse than the original by about 4x. But from the simple -flto -march=native speedup I assume a lot more speed is potentially on the table.

---

• jasonfi (orginal) [2021-10-25T16:34:50+02:00] view original

  The benchmark performance is often proportional to the effort spent optimizing for that benchmark(s). I don't think the Techempower Benchmarks are a priority for the Nim team and community, mainly because there is so much going on already.

---

• kobi (orginal) [2021-10-26T15:37:34+02:00] view original

  this brings the point of running & benchmarking the Nim test suite every build, and noticing regressions in performance.

---

• Jaap (orginal) [2021-10-26T15:58:14+02:00] view original

  https://github.com/TechEmpower/FrameworkBenchmarks/discussions/6552

  apparently a cpu bug workaround caused a big dip in the benchmarks between round 18 and 20

---