I'm comparing the performance of various hashing modules and it appears that secureHashFile has memory leaks. Or am I simply using it incorrectly?
Here's a comparison between nimSHA2 and std/sha1:
# Compiled on a Windows 10 machine with:
# nim c -d:nimTypeNames sha.nim
# Nim Compiler Version 1.2.0 [Windows: amd64]
import std/sha1
import nimSHA2
proc sha256(fileName : string) : string =
const blockSize: int = 1024 * 1024
var file : File
var sha = initSHA[SHA256]()
try:
file = open(filename, FileMode.fmRead, blockSize)
var buffer = newString(blockSize)
var bytesRead = file.readChars(buffer, 0, blockSize)
while bytesRead > 0:
sha.update(buffer.substr(0, bytesRead - 1))
bytesRead = file.readChars(buffer, 0, blockSize)
finally:
if file != nil:
close(file)
sha.final().toHex
echo "Calculating SHA256"
echo sha256("Fedora-Workstation-Live-x86_64-32-1.6.iso") # 1.9GB file
dumpNumberOfInstances()
echo "Calculating SHA1"
echo $secureHashFile("Fedora-Workstation-Live-x86_64-32-1.6.iso")
dumpNumberOfInstances()
echo "Press enter to exit"
discard stdin.readLine()
Output:
Calculating SHA256
4D0F6653E2E0860C99FFE0EF274A46D875FB85BD2A40CB896DCE1ED013566924
[Heap] string: #40; bytes: 2197217
[Heap] WideCStringObj: #0; bytes: 880
[Heap] total number of bytes: 2198097
[Heap] allocs/deallocs: 1921/1876
Calculating SHA1
A936C00AB4BF1B28963823A8E80A828D30812717
[Heap] string: #26; bytes: 1966179519
[Heap] WideCStringObj: #0; bytes: 1760
[Heap] total number of bytes: 1966181279
[Heap] allocs/deallocs: 1951/1918
Press enter to exitCalling them separately results in a post-computation RAM usage of less than 10MB with SHA256 and 1.8GB with SHA1. Calling secureHashFile on large files from a Win32 front-end also results in the type of text corruption that's usually indicative of a memory leak somewhere.
Implementation issue. secureHashFile read the whole file into memory and you wrote a better solution. You may create a PR to nim.