Hello,
Is there an opposite to the tags pragma or some way to simulate it? I'm thinking about something like proc function(params...) {.notTags: [MyEffect].} which would accept any effect but MyEffect.
I consider the effect system itself to be a mistake.
Blog post needed
It's discouraging and demotivating when you call a language feature "a mistake" without elaboration. Should I stop using it? Will it be eventually deprecated? I thought it was a core feature, it is even documented in the Mastering Nim book without any caveats.
Because of remarks like this, except for orc/arc, I have been constraining myself to the features that are dog-fooded by the compiler.
Well it's really hard to do without it and keep all its benefits, so relax nothing will happen to it anytime soon. (Except maybe further improvements if we receive PRs.)
Having said that, here are its downsides:
Just model effects as ordinary parameters. Maybe add a required backend optimization so that parameters of type .effect are always optimized out.
Last year I too wondered about a way to forbid certain effects, and Araq gave a similar suggestion of passing permission parameters around instead. I went and played with this idea a little, but I couldn't find a solution which had zero runtime overhead and would still compile. So, yeah, a way to have "empty" types that get optimised out would be nice for that use case.
I consider the effect system itself to be a mistake.
Do you feel this way about the whole effects system including as it is used for exceptions? I once observed that {.raises:[].} is an elegant solution to a problem that can happen with results or "checked exceptions", where somebody gets lazy and swallows an error because the language forced them to deal with it. The fact that you were able to eliminate this friction while still giving program authors a way to guarantee every error has been accounted for, is remarkable to me!
Do you feel this way about the whole effects system including as it is used for exceptions?
Yes and no. You're right that it works beautifully for exception tracking but I keep looking for something better.
I went and played with this idea a little, but I couldn't find a solution which had zero runtime overhead and would still compile.
In Nimbus we tried to use static enums to represent the validation flow of incoming data:
type ValidationStatus = enum
None
GossipValidated # Network propagation rules were checked
ConsensusValidated # Blockchain protocol rules were checked
Trusted # All is well
This works well at the proc level, however we hit a roadblock when we wanted fields to use distinct type type TrustedAttestation = distinct Attestation like so:
type
Attestation = object
TrustedAttestation = distinctAttestation
Block[V: ValidationStatus] = object
when V == Trusted:
attestation: TrustedAttestation
else:
attestation: Attestation
the issue was for serializers, working with when branches at the type level in macro tends to raise internal compiler errors.
A workqaround is using generic procs and
type Block = object
attestation: Attestation
type GossipValidatedBlock {.borrow:`.`.} = distinct Block
type ConsensusValidatedBlock {.borrow:`.`.} = distinct Block
type TrustedBlock = object
attestation: TrustedAttestation
type AnyBlock = Block or TrustedBlock
but well it's not an "effect" system anymore.
This is not how it works: In order to use/create a ptr UncheckedArray you need the cast
And importc and emit. Also those can be hidden in external dependency and not directly in your code base.
pointer derefs are irrelevant here too
Pointer derefs are not the root cause of unsafe memory access but they can result in undefined behaviour. IMO, having undefined behavior is a problem and I would find it useful if the compiler could help reduce those or mark them clearly.
An example that makes a bad memory access through an indexing proc that actually implements bounds checking :
import std/rtarrays
import arraymancer
proc main() =
var y = initRtArray[float](4)
var x = fromBuffer[float](y.getRawData(), [15])
echo x[10] # Bad memory access -> UB
EDIT: as if these are the only things you can do which are "unsafe" when really you can do anything asm can}
I am not convinced by the argument that "Unsafe is useless if it does not catch 100% of the possible unsafe things".
Detecting potential UB compile time can help reduce bugs / write better (or more tested) code.
Sure, the effect system may not the best way to do it because it adds too much complexity. Maybe tags are better, or using on an external tool like DrNim or Nimsuggest to give this information to the develop is easier.
But come on, do manually grepping through source code, stdlib and every dependency used the best Nim can do ? Or should developer give up entirely on being able to detect unsafe code the moment they use a Nimble package ?
But come on, do manually grepping through source code, stdlib and every dependency used really the best Nim can do?
As opposed to what? Grepping for unsafe instead? And yes, I actually did code reviews of external packages and do search for cast and addr in order to find problems. It's not hard and it's not perfect but you never gave us any viable alternative either.
"In Rust there is 2 keywords less to search for, all things considered" is not a serious argument.
Or should developer give up entirely on being able to detect unsafe code the moment they use a Nimble package?
Pure hyperbole. In the Python eco-system there is plenty of foreign C and C++ code too and you can use libffi to access even more. How come this eco-system thrives? Why is it not discussed over there? Has the Python community "give up entirely on being able to detect unsafe code"?
> Doesn't seem "very clear" to me.
Arraymancer's insanely low level API is not Nim's fault...
To be fair, fromBuffer is simply there for interop with other libraries, e.g. when accessing a numpy array. It's pretty clear that using it is highly unsafe. I think Clonkk mainly wanted to highlight (using valid code) that that an operation that should appear to be safe can easily be obfuscated by layers on top.
"In Rust there is 2 keywords less to search for, all things considered" is not a serious argument.
Rust Unsafe covers more use case than addr and cast and also provide CT checks for undefined behavior, but you know this.
This discussion is pointless if you don't see how having to grep the entirety of ~/.nimble/pkgs as well as the stdlib to potentially find undefined behavior between different Nimble package is inefficient.
Tracing unsafe scope is not a necessity and we can get by without it. It's just nice to have some ergonomics tools when dealing with this sort of stuff but it's not required. And in any case, this isn't gonna be the "make or break" feature for Nim.
I think Clonkk mainly wanted to highlight (using valid code) that that an operation that should appear to be safe can easily be obfuscated by layers on top.
Pretty much spot-on. The point being that relying on "grep" isn't practical when dealing large project with lots of external dependency.
why can these operations not be tagged with Unsafe or something similar?
As far as I know, cast, importc, emit are defined in compiler code and not in stdlib code; so it would requires modifying the compiler itself (and as you can see not everybody is convinced that it could be useful).
If those construct would be wrapped in a system_unsafe.nim (with addr for good measure), then I assume it would become possible to add a tag without modifying the compiler; but I do not know the impact / cost of wrapping such compiler construct.
I do not recall once suggesting that one should grep code.
Yes, I was replying to multiple people at once, just the reply feature that's awkward.
My point is with forbid now existing the compiler can be modified and features can be added which enable unsafe code to be detected by the compiler.
If modifying the compiler is on the table, then yes you can do that; but it was already possible through the effect system. Using tags instead of effect to implement it isn't the blocking point.
"Safety" is a mindset, not a language feature or concept. C# started with the "unsafe" keyword, then Rust came up and they use it as a selling argument. Just look at the kind of people it attracts :/
There is no "Safety"