In other languages, testing libraries exist which allow you to "mock" a function (see https://jestjs.io/docs/mock-functions).
Mocking a function means swapping out a function your program would normally call, with another function. E.g. it would find all invocations of foo() and swap it out with a test function you've written fooMock()
This is very powerful for testing purposes, and is why I am trying to replicate this behavior.
Elegantbeef and I have been talking about this in chat for a while, and have been testing solutions such as using (or abusing) TRMs with little success.
import std/[random]
randomize()
proc genString*(length: int): string =
for i in 0 ..< length:
result &= rand('a'..'z')
import strgen
mock(strgen.genString, proc(length: int): string = "foobar")
assert strgen(12).len == 6
mock(strgen.genString, proc(length: int): string = "foobarbaz")
assert strgen(12).len == 9
You'll notice that strgen would normally output strings with a length of 12 here, but that changes because of the mocks. These are also able to be declared and changed at runtime.
I'm hoping some members of the community could help brainstorm a way for this to be achieved. Some of the solutions we've thought of so far have greatly restricted the desired flexibility of "mocking", or would require compiler changes.
There is no good way to "mock" things except for turning the call you want to mock into using a proc variable so that you can override it.
Having said that, old-school functional/procedural programming doesn't need much mocking. There is data and there is code, in order to test the code setup test data. If setting up test data is too hard that means your data structures are in poor shape. ;-)
Soo, uuhhh, I was just bored on a train. And "I have no idea what I'm doing" applies rather heavily.
But... I just ported: https://nullprogram.com/blog/2016/03/31/ to Nim to try it out for this. To my utter surprise it actually works. I'm pretty sure it's not a good idea to use this for testing, but it's fun, haha.
import posix
proc foo(s: string, x: int) =
echo "Hello ", s, " id ", x
proc bar(s: string, x: int) =
echo "Hax0r you! ", s, " id ", x * 123
type
Instr {.union.} = object
bytes: array[8, byte]
value: uint64
proc hotpatchImpl*(target, replacement: pointer) =
# YOLO who needs alignment
#doAssert (cast[ByteAddress](target) and ByteAddress(0x07)) == 0
var page = cast[pointer](cast[ByteAddress](target) and (not 0xfff))
doAssert mprotect(page, 4096, PROT_WRITE or PROT_EXEC) == 0
let rel = cast[ByteAddress](replacement) - cast[ByteAddress](target) - 5
var instr = Instr(bytes: [0xe9.byte,
(rel shr 0).byte,
(rel shr 8).byte,
(rel shr 16).byte,
(rel shr 24).byte,
0, 0, 0])
cast[ptr uint64](target)[] = instr.value
doAssert mprotect(page, 4096, PROT_EXEC) == 0
template hotpatch*(target, replacement: untyped): untyped =
hotpatchImpl(cast[pointer](target), cast[pointer](replacement))
foo("world", 1)
hotpatch(foo, bar)
foo("world", 1)
import strutils
proc myParse(s: string): int = 1337
hotpatch(parseInt, myParse)
echo "12".parseInt()
(or see here that it actually works:
Easy peasy! :D
import std/[random, sugar]
randomize()
proc genString*(length: int): string =
for i in 0 ..< length:
result &= rand('a'..'z')
assert genString(12).len() == 12
template mock(f: proc; m: untyped) =
template mockImpl{f}(): untyped = m
mock(genString):
(x:int) => "foobar"
assert genString(12) == "foobar"
In all seriousness, even though this is a hack, it won't work for your use case, as TRM do not obey scoping rules so each following call to mock will rewrite all previous.
TBH, I don't know why Nim doesn't allow shadowing. I get for procs it will probably clash with overloading, but for mere variables it's a good practice to reuse transitory names or "lock" mutability by redefinition (var x: int; mutate(x); let x = x;).
TBH, I don't know why Nim doesn't allow shadowing.
I like shadowing very much and we should indeed go the full way. The reason why Nim doesn't allow it is historical. Back then it wasn't as clear cut that it's a good idea and even today many oppose it ("it is confusing / error prone").
I've recently tried the pattern of using patchFile for mocking in unit tests and it worked really well for my use case.
The pattern is in a unit test folder you provide a config.nims file using patchFile to override certain modules. Note this will override these modules for all tests in this folder (AFAICT).
This unit testing is for a pure-Nim simple object store called nvs_config_obj.nim which itself uses a Nim-ified wrapper nvs.nim that provides Non-volatile Storage (NVS) on Zephyr RTOS. I wanted to test both nvs_config_obj and nvs modules from the C API layer up and decided that was the best layer to mock. There are two modules that provide the C FFI for the low level C Apis from the Zephyr RTOS (znvs.nim and zflash.nim).
Here's how it worked out. First the config.nims setup to override the znvs and zflash modules:
switch("path", "$projectDir/../../src")
patchFile("nephyr", "znvs", "znvs_mock")
patchFile("nephyr", "zflash", "zflash_mock")
Next in znvs_mock.nim in the same test folder you can override any api's you're testing. In this case some C read/write APIs using a Table with some byte arrays was sufficient:
# ...
type
nvs_fs* = object
# mock the C struct `nvs_fs` with our custom one
data*: Table[uint16, array[128, byte]]
proc nvs_write*(fs: ptr nvs_fs; id: uint16; data: pointer; len: cint): cint =
# mock nvs c write function, only up to 128 bytes
var buf: array[128, byte]
copyMem(buf[0].addr, len.unsafeAddr, 1)
copyMem(buf[1].addr, data, len)
fs.data[id] = buf
proc nvs_read*(fs: ptr nvs_fs; id: uint16; data: pointer; len: cint): cint =
# mock nvs c read function, only up to 128 bytes
var buf = fs.data[id]
copyMem(result.addr, buf[0].addr, 1)
copyMem(data, buf[1].addr, len)
Now my unit tests are testing the main nvs.nim which wraps some Zephyr RTOS bits, but I just call and use the nvs.nim module as normal in the unit tests:
include nephyr/extras/nvs_config_obj # do an include to access any private procs/vars
suite "nvs basic config object":
test "basic store":
var nvs = initNvsMock[NvsConfig]()
var settings = newConfigSettings(nvs, ExampleConfigs())
settings.values.dac_calib_gain = 1111
settings.values.dac_calib_offset = 2222
var fld3Val = nvs.read(fld3, float32)
# etc...
There's some modifications to how you import modules in nvs.nim to help make it easier to "intercept" certain modules. In this case it let's me override the C header imports it tries to do otherwise which is also nice.
I found a bit of trickiness with the raw NVS constructor vs the high level Nim NVS module. I resolved this by passing in the high-level NVS object as a generic type. This allowed me to mock only the lower level znvs.nim module but unit test the higher level NVS wrapper.
proc initNvsMock*[T](): T =
result = T()
discard nvs_init(result.fs.addr, "flash_mock")
You can find the config.nims and the mocked zephyr rtos nvs api's here . The actual unit tests.
Note the method above doesn't let you change things at runtime. Though perhaps you could use the patchFile pattern with some when statements and some tedious usage of include you could potentially mock the core api's differently by setting different var opt {.compileTime.} values, etc. But that'd be annoying to write mocks in one file and tests in another but it could work.
Another pattern would be something like:
# file: fooMock.nim
template mockedFooTmpl(a, b: untyped): untyped =
mockedFoo(a, v)
proc foo(a: int, b: int): int =
result = mockedFooTmpl(a, b)
Then in your unit test:
# file: testFoo.nim
suite "basic tests":
test "essential lies":
proc mockedFoo(a, b: int): int =
result = 34 # break api
check foo(10, 100) == 34
test "essential truths":
proc mockedFoo(a, b: int): int =
result = a * b
check foo(10, 100) == 1_000
I've done something like the above and it got fiddly with compiler when mocking template's of templates... but for simpler proc's it seemed to work well enough. Though it wasn't as split out into mocked modules.
Really, I'm not 100% on understanding the scoping rules of the mockedFooTmpl above. The manual points out mixin statements to help say a symbol like mockedFoo would be "open". Perhaps it could be put into a library to ensure it was setup so it'd be easier to do and the open vs closed symbols could be more reliably handled.
From what I'm seeing here so far, I think the approach from Vindaar looks like the one that seems to make the most sense. When you unit-test a proc, you'd want to be able to test various scenarios, that kind of makes being able to hotswap proc-definitions at runtime mandatory.
I wonder how C properly deals with it as the most prominent procedural language. Sadly I know next to nothing about C, so the articles (http://locklessinc.com/articles/mocking/ ) and libs (CMock) I've found are only partially legible to me. What I understand from them though is that mocking procs like that appears to be non-trivial, especially if you don't want to have to change your source-code just to be able to run tests.
Besides Vindaar's suggestion, the only thing I can think of is putting all imports behind some kind of compile-time flag so you can modify said import paths to take the modules from a directory where you define "mock"-modules instead of their actual place.
I have arrived at a solution thanks to ElegantBeef and Vindaar!
The core idea is that you can use the power of pragmas/macros to rewrite a function definition so that it is instead an anonymous proc assigned to a mutable variable!
What this allows you is then to simply reassign the proc in the mutable variable with your mock-proc and you're good to go! I'm currently in the process of putting that idea into its own package with utils that take care of various things surrounding that: https://github.com/PhilippMDoerner/mockingbird