No big deal, but it might be of interest to some of you. For a project, I needed a configuration format that could represent hierarchies and was easy for humans to write.
I initially looked at KDL (there are two implementations for Nim, see https://github.com/Patitotective/kdl-nim (KDL 1.0) and https://github.com/greenm01/nimkdl (KDL 2.0), but found it too complex for my needs. Then, by chance, I stumbled upon Simon Ser's scfg project, which implements a simple, line-based language. There's also a Nim implementation for it (see https://codeberg.org/xoich/nim-scfg), but I haven't tried it. I wanted one file that I could simply drop into a project.
To cut a long story short, here is scfg-nim
Examples how scfg looks like:
server {
listen 80
server_name example.com www.example.com
location / {
root /var/www/html
index index.html index.htm
}
location = /robots.txt {
allow all
log_not_found off
access_log off
}
}
train "Shinkansen" {
model "E5" {
max-speed 320km/h
weight 453.5t
lines-served "Tōhoku" "Hokkaido"
}
model "E7" {
max-speed 275km/h
weight 540t
lines-served "Hokuriku" "Jōetsu"
}
}(Already you used some goofy syntax location = /robots.txt I cannot interpret reliably.)
You've probably never configured nginx before? ;)
The aforementioned examples are meaningless in themselves; it is up to the implementation to interpret them. scfg only recognizes directives with a name (string), optional parameters (each is a string), and optional children (directive).
The following example:
$x = 42
$found_answer = false
if $x == 42 {
$found_answer = true
}is:
Version 0.2.1 now includes an event-based API in addition to the "normal" API, which reads the entire configuration file.
Previously, I read the configuration files and then traversed the tree to interpret the configuration and flag invalid values, etc. The event-based API now eliminates the need for this re-traversal and allows me to react directly to events.
The function that builds a tree of directives also internally utilizes the event-based API (previously, the tree was built recursively), thus removing the previous limit of 1000 nested directives. While this likely has no practical impact (who creates such a deeply nested configuration file?), it's a nice side effect.
With this API, it would also be easy to validate configuration files without having to read them completely.
So, if you want to read your typical multi-gigabyte configuration files with minimal memory overhead, the event-based/streaming API could be a good option. ;)
Here's an example of how to use the API (full source code in the README)
let stream = new_string_stream(server_config)
var
servers: seq[ServerConfig]
depth = 0
in_server = false
in_location = false
for event in parse_scfg(stream):
case event.kind:
of evt_start:
inc depth
if not in_server and event.name == "server":
in_server = true
servers.add ServerConfig()
continue
if in_server and not in_location:
case event.name:
of "location":
in_location = true
servers[^1].locations.add LocationConfig(
access_log: true,
exact_match: event.params.len > 0 and event.params[0] == "=",
path: if event.params.len > 0: event.params[^1] else: ""
)
of "listen": servers[^1].port = event.to_uint()
of "server_name": servers[^1].names = event.params
else: error("Unknown directive: " & event.name)
elif in_location:
case event.name:
of "root": servers[^1].locations[^1].root = event.to_str()
of "index": servers[^1].locations[^1].index = event.params
of "allow": servers[^1].locations[^1].allow = event.to_str()
of "log_not_found": servers[^1].locations[^1].log_not_found = event.to_bool()
of "access_log": servers[^1].locations[^1].access_log = event.to_bool()
else: error("Unknown directive: " & event.name)
of evt_end:
dec depth
if in_location and depth == 1:
in_location = false
elif in_server and depth == 0:
in_server = false
With the current version (fully backwards compatible) the example gets even shorter: The end events carry an information if a block was closed or not.
Previously, the end user had to consider whether the parser was within a block and manage a more or less complex stack. This can now be solved directly via the API.
let stream = new_string_stream(server_config)
var
servers: seq[ServerConfig]
in_server = false
in_location = false
for event in parse_scfg(stream):
case event.kind:
of evt_start:
if not in_server and event.name == "server":
in_server = true
servers.add ServerConfig()
elif in_server and not in_location:
case event.name:
of "location":
in_location = true
servers[^1].locations.add LocationConfig(
access_log: true,
exact_match: event.params[0] == "=",
path: event.params[^1]
)
of "listen": servers[^1].port = event.to_uint()
of "server_name": servers[^1].names = event.params
else: error("Unknown directive: " & event.name)
elif in_location:
case event.name:
of "root": servers[^1].locations[^1].root = event.to_str()
of "index": servers[^1].locations[^1].index = event.params
of "allow": servers[^1].locations[^1].allow = event.to_str()
of "log_not_found": servers[^1].locations[^1].log_not_found = event.to_bool()
of "access_log": servers[^1].locations[^1].access_log = event.to_bool()
else: error("Unknown directive: " & event.name)
of evt_end:
if event.has_block:
if in_location:
in_location = false
elif in_server:
in_server = falseI've added an example that demonstrates the simplicity and possibilities of scfg. Since scfg only recognizes directives with parameters and optional blocks, it's up to the implementation to interpret the scfg documents.
This makes it easy to, for example, interpret some constructs for declaring variables.
The implementation uses
<identifier> = <value>to declare a variable. However, you could just as easily use := or :, or any other string, like is-declared-as.
In this example, the application logic remains the same; the variables are resolved during parsing, so the application neither sees the variable declarations nor is responsible for resolving them.
The implementation doesn't impose any restrictions on what is allowed as an identifier, so
$ringo = "Ringo Starr"
ringo = "RIngo Starr"are valid, though not identical, variables. Every directive name is a valid identifier.
However, it would be easy to restrict the variables to specific identifiers, for example, to only allow $identifier.
The example program reads the input from stdin and writes the event stream it encounters back to stdout.
I find scfg very flexible and easy to build upon with its simple syntax.
It would also be possible to incorporate simple conditions so that a different configuration is read on a laptop than on a desktop computer, while both can use the same configuration file.
A few examples (also in the repository):
Input:
$ringo = "Ringo Starr"
the-beatles "Paul McCartney" "John Lennon" "George Harrison" $ringo
plays-drums $ringoOutput:
the-beatles "Paul McCartney" "John Lennon" "George Harrison" "Ringo Starr"
plays-drums "Ringo Starr"Input:
$border = {
left-margin 10
right-margin 10
background #000
foreground #fff
}
widget-a {
name "A widget"
border $border
padding 0 10 0
}
widget-b {
name "Another widget"
border $border
}Output:
widget-a {
name "A widget"
border {
left-margin 10
right-margin 10
background #000
foreground #fff
}
padding 0 10 0
}
widget-b {
name "Another widget"
border {
left-margin 10
right-margin 10
background #000
foreground #fff
}
}And, as said, the implementation is very lenient what's accepted as identifier…
Input:
'Charlie Watts' = "Ringo Starr"
the-beatles "Paul McCartney" "John Lennon" "George Harrison" "Charlie Watts"
plays-drums 'Charlie Watts'Output:
the-beatles "Paul McCartney" "John Lennon" "George Harrison" "Ringo Starr"
plays-drums "Ringo Starr"Input
42 = the meaning of life
what is 42Output:
what is the meaing of lifeThis post isn't so much about using the implementation as a role model, but rather about praising the simplicity and flexibility of scfg. :)
I hadn't imagined "wasting" so much time on it. ;)
Anyway, since the event-based API is available, the ability to create a data structure of scfg blocks and directives isn't particularly useful for me.
This option is no longer available in version tagged 0.3.0, and the functionality has been moved to an example in the examples directory. This reduces the deserializer to just over 100 lines of code.
The last version to include both the event-based and "traditional" APIs is tagged with 0.2.4.