As for my first intuition, I agree with Araq here...
In fact, I have heard of other people experiencing first-run slowness as Windows Defender (or similar) checks out a dynamically linked library.
So my question is: does it get better if you run the same executable twice? I think you may be running the build each time. Try building once, then running the EXE directly several times with an unchanged DLL and unchanged EXE. Does that improve the time?