Hi all,
Apologies if this is a noob question, I'm quite new to Nim.
For a program, I'm trying to Base64-decode a binary blob that was encoded using Python into a Nim byte array. Since the binary is an embedded executable, it will contain non-printable characters and cannot be stored in a string object. What I'm trying to do on a high level is as follows:
import base64
var b64test = "TVqQAAMAAAAEAA=="
# bytearray(b'MZ\x90\x00\x03\x00\x00\x00\x04\x00') encoded in Python
# should become something like let data : array[10, byte] = [byte 0x4D,0x5A,0x90,0x00,0x03,0x00,0x00,0x00,0x04,0x00]
b64result = decode(b64test)
echo repr(decode(b64result))
This however yields the following error Error: unhandled exception: Invalid base64 format character '' (ord 144) at location 2. [ValueError]. This error as I understand it is because the decoder encounters an unprintable character, which is expected within the context of my program. Any way to work around this and decode this Base64-blob to a byte array such as the one provided in the code?
Side note: If it is possible to easily use encryption (e.g. AES) with a statically defined key instead of Base64-encoding to store the object and later decrypt it, that's an acceptable (or even preferred) alternative for the purposes of my program. I was however running into similar issues with the crypto libraries that I found.
Thanks a lot in advance for any guidance!
I do not recommend the stdlib/base64 module.
It has an unstable API and conflates strings and byterrays. Instead you can use stew/base64 https://github.com/status-im/nim-stew/blob/master/stew/base64.nim
Regarding static AES, I think nimcrypto's Rijndael works at compile-time https://github.com/cheatfate/nimcrypto/blob/master/nimcrypto/rijndael.nim .
Historically there wasn't a way to replace and evolve the stdlib.
Fusion (https://github.com/nim-lang/fusion) improves the situation a bit by making it a staging area for candidates for stdlib.
But I concur, an official way to evolve the standard library while making it as painless as possible a transition is needed. Note that some modules like base64 were already identified has having an API that is likely to be reviewed further and this is mentioned in their docs.
Nim strings CAN handle arbitrary bytes. Also your example works for me:
import base64
let b64test = "TVqQAAMAAAAEAA=="
let b64result = decode(b64test)
assert b64result.len == 10
Now (that you know that nim strings can store binary data), do you still need byte arrays?
Now, that you know, that nim strings can store binary data, do you still need byte arrays?
Integers can also handle pointers, that doesn't mean we should use int object of RootObj. We have a type system powerful enough to avoid mixing Miles and Meters, we don't need to be bound by C. For example, binary blobs are semantically significantly different from ASCII or UTF-8, most strutils is useless for binary blob for example.