I'm trying to build Nim library for plotting charts and data tables. And I can't make the Nim DSL to be clean compact.
Is there a better way to do it? Here's the table I'm creating, some portfolio
The TypeScript DSL to build such table, very minimal, very clean, and fully type safe with compile time validations and IDE autocomplete.
import type { TableOptions } from "./table/scheme"
const options: TableOptions = {
order: [["mv_usd", "desc"]],
columns: [
{
id: "mv_usd",
type: "number",
format: { type: "line", ticks: [1000] }
},
{
id: "tenor",
type: "number",
format: { type: "line", ticks: [180, 360, 720] }
},
{
id: "right",
type: "string"
},
{
id: "symbol",
type: "string"
},
{
id: "contract_id",
type: "string",
format: { type: "string", small: true }
}
],
rows: []
}
I tried to convert it to Nim, here's the Nim data scheme, pl0t.nim
But the result is not very clean.
import pl0t # https://github.com/al6x/pl0t/blob/main/api/nim/pl0t.nim
let options = TableOptions[seq[float]](
order: @[("mv_usd", PlotOrder.desc_e)].some,
columns: @[
PlotTableColumn(
id: "mv_usd",
`type`: "number",
format: FormatOptions(
`type`: PlotFormatType.line_e,
ticks: @[1000.0].some
).some
),
PlotTableColumn(
id: "tenor",
`type`: "number",
format: FormatOptions(
`type`: PlotFormatType.line_e,
ticks: @[180.0, 360.0, 720.0].some
).some
),
PlotTableColumn(
id: "right",
`type`: "string"
),
PlotTableColumn(
id: "symbol",
`type`: "string"
),
PlotTableColumn(
id: "contract_id",
`type`: "string",
format: FormatOptions(
`type`: "string",
small: true.some
).some
)
].some,
rows: @[]
)
Is there way to improve it?
Instead of type use the word kind.
What you have here is not a DSL at all, you create a data structure that is then interpreted somewhere. That's unnecessarily convoluted, you should map the DSL to imperative code directly. Via templates and macros.
I found one more way. And used the std/json.%* (I altered it a little bit to avoid the need for quotes for object keys and called jo). So the new Nim API:
plot("/portfolio.json", jo {
order: [["mv_usd", "desc"]],
columns: [
{
id: "mv_usd",
type: "number",
format: { type: "line", ticks: [1000] }
},
{
id: "tenor",
type: "number",
format: { type: "line", ticks: [180, 360, 720] }
},
{
id: "right",
type: "string"
},
{
id: "symbol",
type: "string"
},
{
id: "contract_id",
type: "string",
format: { type: "string", small: true }
}
]
})
Couple advantages:
P.S.
About using macro for API... Templates are cool. But macros are just too hard, it's like learning a whole new language, I can't use it.
Yes. I do use Vega-Lite too, it is awesome. But I don't like DSL (I used DSL heavily in Ruby, but don't use it anymore).
After working with TypeScript kinda homoiconic-like data format, with 1-to-1 match with language types, fully type-safe and 100% IDE autocomplete and with literal types. It's superior to DSLs. You have instant DSL for pretty much anything, with zero code, instantly, by just defining a type.
macros are just too hard
One of the big benefits of Nim is the awesome metaprogramming, maybe read about:
It would be really great if there were a karax-style DSL for plotting. Even better if it supported multiple backends like vega/vega-lite.
Who really wants something as verbose as karax or raw vega-lite as a plotting tool though?
This:
https://github.com/Vindaar/MetaPlot
was an attempt at something like this. The idea was a helper binary that is called at CT for a schema check of the plotly / vega-lite input. The usage would have been so verbose though and the additional binary to check the schema so complicated, that I dropped the idea. A much better idea to actually just write a plotting library with a sane API that can be checked at CT naturally.
For a different approach to CT checks for such a thing, in my LatexDSL:
https://github.com/Vindaar/LatexDSL
I do something similar to karax. I have a huge enum of all allowed identifiers:
https://github.com/Vindaar/LatexDSL/blob/master/src/latexdsl/valid_tex_commands.nim
which can be "easily" extended (there's a mini nim program in the repo that you hand a latex style file and it extracts all tex macros and dumps it into an enum). Works fine to generate somewhat "CT safe" latex code. A use case is here:
https://gist.github.com/Vindaar/545cf13fb09d75843ea0eef0dec1dae0
But back to plotting: vega-lite is great, I agree. Hence why after dropping the MetaPlot ideas I started writing ggplotnim with the full intention from day one to have a vega-lite backend. It's not complete, but it works rather well already:
https://github.com/Vindaar/ggplotnim/blob/master/recipes.org#simple-vega-lite-example
There's a significant difference between Vega-Lite and other plotting libraries like Matlab, matplotlib etc.
Vega-Lite is visualisation grammar, a set of primitives that could be combined and composed in many different ways. Other plot libs are predefined boilerplates, want scatter - use template for scatter, want historgram - you need different template, there are less reusability and universality.
Vega-Lite in its raw form is tool low-level. Usually a higher level wrapper wrapper used, like Altair for Python.
I just realised there's one more reason why I would like to use pure data API instead of specialized DSL. Just a nice and compact way to build data structures.
Because if you work with plots and data analysis, you probably spent at least 50% of the time with Python and/or JS/TS. And you don't want to learn and use 3 different plot libraries, with different syntaxes in those 3 languages.
Vega-Lite again shine here. Because it has data-driven API. And the data - arrays, objects (maps/tables) etc. look pretty much the same in all languages. So you are able to use almost 1-to-1 ploting code in Nim, Python, TypeScript.
This is a bit of an aside, but you could implement this whole TS feature with Nim macros. Rather than writing a DSL for plotting, you could write a macro that takes a schema and produces types and a DSL using a pretty similar format to TS. It might be a decent amount of work, and I'm not sure if you could match TS 100%, but you could get pretty close.
This one of the things I like best about Nim. I don't need the core team to anticipate my needs, or even care about my use-case. If it's important to me, I can do it myself, and do so in a way that feels integrated into the language (nice error reporting, etc.). Writing Nim macros isn't exactly trivial, but it's certainly easier than adding new language features.
Yehuda Katz wrote about this from a Ruby vs Python standpoint ages ago, explaining why Rails showed on on Ruby rather than Python. Nim metaprogramming is harder than Ruby metaprogramming, but also much more powerful, and with fewer opportunities to shoot yourself in the foot. It's a very useful tool.
+1 for using macros. I'm currently doing something like this. I'm trying to integrate Plotly with a Jupyter Notebook-like IDE I'm making. The DSL so far looks something basic like:
plot LineGraph:
x:[0,1,2,3,4,5]
y:[0,1,2,3,4,5]
You can abstract away A LOT of code.