nimforum mirror - Compile time code optimization.

Nycto (orginal) [2023-11-12T18:12:44+01:00] view original

I've got a project that makes heavy use of macros and build time code-gen. The build has slowed down by a few seconds, so I'm spending some time optimizing the bottlenecks. However, I'm struggling to create a mental model of what is slow within the VM. For example, this change sped up compile times by about 12%. The specific thing that seems to have been faster was directly using an iterator instead of creating an anonymous proc.

This was slow:

let cases = details.createArchetypeCase(newDotExpr(entityIndex, ident("archetype"))) do (fromArch: auto) -> auto:
            if lookup.canCreateFrom(fromArch):
                details.buildArchetypeLookup(lookup, fromArch)
            else:
                quote: return none(`tupleType`)

And this was faster:


        var cases = nnkCaseStmt.newTree(newDotExpr(entityIndex, ident("archetype")))
        for (ofBranch, archetype) in archetypeCases(details):
            if lookup.canCreateFrom(archetype):
                cases.add(nnkOfBranch.newTree(ofBranch, details.buildArchetypeLookup(lookup, archetype)))
        
        let noneResult = quote: return none(`tupleType`)
        cases.add(nnkElse.newTree(noneResult))

I'm guessing it's because the callback has to create a lambda, but I really am just guessing. I found this by enabling the provileVM compiler flag, picking a file that was listed as slow and then progressively commenting out sections of code until things sped up.

So I guess I have two questions:

Are there any patterns of things generally recognized as "slow" at compile time?

Is there a better way to pinpoint the bottlenecks?

Cheers

Nycto (orginal) [2023-11-12T19:02:51+01:00] view original

Just to document my journey a bit, switching from object to ref object for all my compile time state management sped up my macros by around %57:

https://github.com/NecsusECS/Necsus/commit/8aa68038f56bea718d86014e02c753e255e09b98

Araq (orginal) [2023-11-13T13:10:05+01:00] view original

Fwiw both optimizations that you did are common for runtime Nim as well.

Nycto (orginal) [2024-03-24T01:39:40+01:00] view original

Just thought I would follow up here. Since my ultimate goal was faster builds, I ultimately found that the biggest end-to-end improvements were to simply reduce the amount of code being generated. It’s a bit of a “no kidding” realization, but also a bit opaque at the same time.

For example, this set of changes reduce my generated code size by about 60% on one of my projects:

https://github.com/NecsusECS/Necsus/compare/c4ade2f879e428d1408f7e04a6ba403a9fead8cf...a41fc608efa5972ad4908e9f598d58e6a17f2113

With that, compile times (on my laptop) were reduced by more than 35%.

My strategy was pretty simple — dump the size of the generated files in the Nim cache directory, then fiddle with the generated code to see if I could shrink it.

Mirror of forum.nim-lang.org

10626 :: Compile time code optimization.