# nimble build -d:release=safe
# nim 2.2.4
import std/strformat
proc `!`*[T](a: openArray[T]): seq[T] =
## Turns an *openArray* into a sequence.
##
## This is not as efficient as turning a fixed length array into a sequence
## as it always copies every element of `a`.
newSeq(result, a.len)
for i in 0..a.len-1: result[i] = a[i]
block:
var a: array[1_000_000, int]
var b: seq[int] = newSeq[int]()
for i in low(a) .. high(a):
b.add(a[i])
echo &"{b.len = :}"
let c = !a # there is ok
echo &"{c.len = :}"
let d = @a # there is ok
echo &"{d.len = :}"
block:
var a: array[10_000_000, int]
var b: seq[int] = newSeq[int]()
for i in low(a) .. high(a):
b.add(a[i])
echo &"{b.len = :}"
let c = !a # there is ok
echo &"{c.len = :}"
let d = @a # Segmentation fault(core dumped)
echo &"{d.len = :}"Creating a new sequence from a large array (10,000,000 integers): newSeq works. toSeq works. @ from fails with a segfault.
Creating a new seq using @ from an existing seq of length 10,000,000 works.
This comes up a lot - and not just in Nim. Here's another thread (with links to others). Briefly, even what the default limits are depends upon the sysadmin for some deployed system. I have been cranking mine way up for decades. No idea how common/rare this is on Windows.
A program usually has no idea what the limits currently in force at program start-up are. Often enough "the scales" of problems just come from external data like files / command parameters / the network, etc.. AFAICT, there's just little culture of checking limits - much like C's culture of unchecked array bounds.
Perhaps not as part of the stdlib / compiler runtime for conversion which can stay as is, but maybe just as an addition stdlib call, might it be helpful to add some kind of portable interface using GetCurrentThreadStackLimits on Windows and getrlimit on Unix and NotSure on "other"? This might make it easy for a Nim program to guestimate its needs and if not met by 1.5..2X or more error out with a helpful message. Then, perhaps, we can make a more helpful response to queries like @biran2038's?
Maybe I'm ignorant and something like this already exists, but I did not find GetCurrentThreadStackLimits anywhere except an importc in winim/winim/inc/winbase.nim. Maybe there's some problem on Windows with that working or another name for the API?
Another option (which might already be in wide use?) is using the backend C compiler's sanitizer/etc. modes to isolate failures from more mysteriously blowing these limits out, but a helpful message for an end user that needs to ask an admin to do something is probably not on offer there.
But a is not on the stack, global variables live in the data segment (or in this case, I guess bss). Had a lived on the stack, it would have immediately crashed like this:
proc main =
var a: array[10_000_000, int]
echo a.len # goodbye
main()That brings up the question, what is overflowing the stack. Compiling following code with -d:danger:
var myArray: array[10_000_000, int]
let myArrayCopy = @myArray
stdout.writeLine myArrayCopy.lenyields
typedef NI tyArray__XQpq8SssyrXu6SqZAuwqFg[10000000];
[...]
N_LIB_PRIVATE tyArray__XQpq8SssyrXu6SqZAuwqFg myArray__x_u1;
[...]
N_LIB_PRIVATE N_NIMCALL(void, NimMainModule)(void) {
{
tyArray__XQpq8SssyrXu6SqZAuwqFg colontmpD_;
NimStringV2 colontmpD__2;
NI T2_;
tyArray__nHXaesL0DJZHyVS07ARPRA T3_;
NI T4_;
NIM_BOOL* nimErr_;
nimErr_ = nimErrorFlag();
nimZeroMem(((void*) colontmpD_), sizeof(tyArray__XQpq8SssyrXu6SqZAuwqFg));
colontmpD__2.len = 0;
colontmpD__2.p = NIM_NIL;
nimCopyMem(((void*) colontmpD_), ((NIM_CONST void*) myArray__x_u1), sizeof(tyArray__XQpq8SssyrXu6SqZAuwqFg));
myArrayCopy__x_u6.len = 10000000;
myArrayCopy__x_u6.p = ((tySequence__qwqHTkRvwhrRyENtudHQ7g_Content*) newSeqPayload(10000000, sizeof(NI), NIM_ALIGNOF(NI)));
T2_ = ((NI) 0);
for (T2_ = 0; T2_ < 10000000; T2_++) {
(myArrayCopy__x_u6).p->data[T2_] = colontmpD_[T2_];
}
T4_ = myArrayCopy__x_u6.len;
colontmpD__2 = dollar___systemZdollars_u14(T4_);
if (NIM_UNLIKELY((*nimErr_))) {
goto LA1_;
}
T3_[0] = colontmpD__2;
writeLine__x_u11(stdout, T3_, 1);
[...]So it seems @ first copies myArray to the stack, and then copies the copy to the heap. I don't know why.
Maybe, the backend C compiler optimize out the large array from the stack because the array in your code is 0 filled and C compiler can generate assembly code that just adds 0 to the seq. But when '@' operator is used, such optimization doesn't applied and generate the large array on the stack. In that case, you probably need to read the assembly code generated by the backend C compiler. (If you use GCC, my article might help to generate assembly code https://internet-of-tomohiro.pages.dev/nim/nimruntimecheckoptimize.en).
Or try to assign runtime values (read from stdin, files, etc) to the big array so that no optimization can removes the stack variable.
Nim has a tendency to introduce lots and lots of temporary copies of everything - these slow things down and use up stack space - likely, @ falls into the category of code where nim simply adds a few copies here and there due to bugs / incomplete analysis.
Indeed, the recommendation in such cases is to look at the C code and report an issue with a small example where you understand the copy is redundant but the nim compiler does not.
but I don't know about the complexities involved in that.
proc `@`* [IDX, T](a: sink array[IDX, T]): seq[T] {.magic: "ArrToSeq", noSideEffect.}
This sink annotation here is just a typo, I think.