This is something I grappled with during the development of Pymod.
I wanted to extract double-hash comments automatically from Nim code, so I could output them into Python docstrings generated by Pymod.
I asked on the Nim forums; Araq confirmed that it was not possible at that time. So, I created a little work-around: nim-docstrings. This is what Pymod currently uses.
Recently I discovered that the Nim docgen utility can now write extracted Nim docs (including double-hash comments) to a JSON file on disk: nim jsondoc sourcefile.nim. This might be your best option.
(Unless you change the Nim compiler to retain the comment text in a nnkCommentStmt node. If you do that, please let me know, and I'll use it!)
Is your chat with Araq online, so I can read it & get up-to-speed on the current situation?
Is there a preference for the implementation of keeping comment strings in nnkCommentStmt?
I seem to recall (some time back) that there was disagreement in the Nim community as to whether everyone actually wanted comment strings in their nnkCommentStmt nodes all the time. Is this recollection correct, or have I misremembered?
PS: I'm quite happy to try to implement it. (I've already pondered some of the relevant issues when I was making nim-docstrings, and I've spent more time digging around the Nim compiler source code than your average human.)
Excellent, thank you @Araq! I found your commit: allows macros to access documentation comments on devel branch.
Here's the code I used to test it:
import macros
import strutils
macro extractDocComments(p: expr): stmt =
hint("\n\n")
hint(treeRepr(p))
hint("\n\n")
for i in 0..<p.len:
let n = p[i]
hint("- $1 $2" % [$n.kind, treeRepr(n)])
if n.kind == nnkStmtList:
for i2 in 0..<n.len:
let n2 = n[i2]
hint("--- $1 $2" % [$n2.kind, treeRepr(n2)])
if n2.kind == nnkCommentStmt:
hint(n2.strVal)
result = newStmtList()
proc foo(x: int): int {.extractDocComments.} =
# Here is a non-doc comment 1.
## This is a documentation comment.
##
## Here are more lines of the same documentation comment.
# Here is a non-doc comment 2.
result = x + 1
# Here is a non-doc comment 3.
## Here is more documentation comment.
# Here is a non-doc comment 4.
Here are the relevant lines of compiler output:
test1.nim(12, 9) Hint: - nnkIdent Ident !"foo" [User]
test1.nim(12, 9) Hint: - nnkEmpty Empty [User]
test1.nim(12, 9) Hint: - nnkEmpty Empty [User]
test1.nim(12, 9) Hint: - nnkFormalParams FormalParams
Ident !"int"
IdentDefs
Ident !"x"
Ident !"int"
Empty [User]
test1.nim(12, 9) Hint: - nnkPragma Pragma [User]
test1.nim(12, 9) Hint: - nnkEmpty Empty [User]
test1.nim(12, 9) Hint: - nnkStmtList StmtList
CommentStmt
Asgn
Ident !"result"
Infix
Ident !"+"
Ident !"x"
IntLit 1
CommentStmt [User]
test1.nim(16, 13) Hint: --- nnkCommentStmt CommentStmt [User]
test1.nim(18, 15) Hint: ## This is a documentation comment.
##
## Here are more lines of the same documentation comment. [User]
test1.nim(16, 13) Hint: --- nnkAsgn Asgn
Ident !"result"
Infix
Ident !"+"
Ident !"x"
IntLit 1 [User]
test1.nim(16, 13) Hint: --- nnkCommentStmt CommentStmt [User]
test1.nim(18, 15) Hint: ## Here is more documentation comment. [User]
Hint: [Link]
Hint: operation successful (12989 lines compiled; 0.191 sec total; 18.189MB; Debug Build) [SuccessX]
Feedback: Is there any particular reason why the double-hash ## at the start of each line can't be removed from the start of the line when the line is read & stored? I assume that anyone who does anything with this documentation comment text will immediately want to remove the double-hash anyway...(?)
(I note that in compiler/docgen.nim, there is a lot of checking that the .comment string begins with double-hash.)
Tried to do like this:
elif a.kind == nkCommentStmt: regs[ra].node.strVal = a.comment.split("\n").mapIt(strip(it[2..^0])).join("\n")