We've recently introduced some exception-tracking cleanups in chronos - this change fixes several long-standing and practical issue we've had, namely that exception tracking was not working at all for async functions and instead infected large parts of the codebase with invalid deductions - this is great news when trying to write exception-safe code!
To write code that allows the compiler to deduce exception effects correctly, a few rules must be followed and I thought I'd document them here because they often come up in code reviews - at Status we mostly write exception-free code and enforce it with a top-level {.push raises: [].} in each non-raising module, roughly dividing the world into "this module raises" and "this module doesn't raise" - a convenient granularity to reason about it.
A simplified example of the problem is the following:
type
Callback = proc()
X = object
callback: Callback
proc f(x: X) {.raises: [].} =
x.callback() # Error: can raise an unlisted exception: Exception
The three places in the language where the compiler is unable to deduce effects correctly are: proc types, methods and forward declarations - in all cases, the solution is simple: annotate them with explicit raises information - not only does this tell the compiler what the expectations are - the reader of the code can also immediately tell what level of exception handling the code supports. It turns out that code can often be simplified and optimized - specially callbacks and methods - when it's known not to be raising exceptions - enforcing it with effects helps catching mistakes early and teaching developers about the API, specially during refactoring.
Here's the copy-paste template we use:
type Callback = proc () {.gcsafe, raises: [].}
method f() {.gcsafe, raises: [].}
proc f() {.gcsafe, raises: [].}
When proc types are missing gcsafe and raises, the compiler must assume they might raise an Exception or contain unsafe code - this infects any calls to the proc with this worst-case deduction. The same thing happens with method and forward declarations - when the compiler sees them, it cannot yet examine the implementation and thus assumes the worst case.
For the same reason, in sensitive parts of the codebase that benefit from the additional simplicity, we also add noSideEffect such that the callbacks safely can be used from within a func.
The effect of this is that any time a function that doesn't conform to the raises information is passed to a variable of Callback type, the compiler will highlight that it's not a match, reminding the caller that the code using Callback is not prepared to deal with exceptions.
When proc types are annotated with raises, a concrete procedure raising "more" than the specifier allows can no longer be assigned to a variable of the proc type:
type
Callback = proc() {.raises: [].}
X = object
callback: Callback
proc f(x: X) {.raises: [].} =
x.callback() # No more error - calling callback no longer can raise
# Not too easy to decipher, but "raise effects differ" is the key information here:
# Error: type mismatch: got <proc (){.noSideEffect, gcsafe, locks: 0.}> but expected 'Callback = proc (){.closure.}' .raise effects differ
var x = X(callback: proc() = raise (ref ValueError)())
Using exception tracking like this puts some constraints on what you can do inside a callback - it turns out that most of the time, it is not a real problem, the exception being legacy API that was not designed for exception tracking - this unfortunately includes many modules in the standard library.
Consider for example strformat - it raises a ValueError when the format string is invalid, which in theory could be checked at compile time for constant format strings - nonetheless, it's deduced as raising ValueError and the effect spreads - this of course is bad for everyone involved: performance suffers due to missed optimizations, readers of the code are confused by unintuitive effects and more bloat must be added to disarm the useless effect.
Another common case is table: there are no good ways to check-and-get an item from a Table without double lookups and invalid effects - if a in table: return table[a] is a fairly common construct in nim, but in addition to being slow (two lookups instead of one), it also introduces bloat (code is generated for a KeyError that cannot be raised) and infects the effect system.
We tend to approach such cases by simply disarming the exception effect with a local try/except, preventing tech debt from building up beyond the "current" module - once a sufficient number of such hacks are introduced, we fix the API instead, and can then simplify any calling code as well.