Mirror of forum.nim-lang.org

10781 :: How to await a list of async procedures

• coneonthefloor (orginal) [2023-12-18T12:40:04+01:00] view original

  I am working on a tool that scrapes the content of a sitemap.

  I have an async procedure getEntries that I call for each sitemap entry in a sitemap index.

  In my current code I await the result of each call to getEntries, code below.

  for sitemapIndexEntry in getSitemapIndexEntries(sitemapContent):
        await getEntries(sitemapIndexEntry.loc)
  Run

  Is there a way that instead of waiting for each call, I could add the call to a sequence and await the completion of all calls. This would allow for the requests to run in parallel, instead of one at a time.

  Basically the equivalent of Promise.all in JavaScript.

  Please let me know if there is a more dogmatic approach to the one I have described.

---

• amadan (orginal) [2023-12-18T12:58:46+01:00] view original

  You can add all the futures to a sequence and then call all on it like

  var futs: seq[Future[Something]]
  for sitemapIndexEntry in getSitemapIndexEntries(sitemapContent):
        futs &= getEntries(sitemapIndexEntry.loc)
  let entries = await futs.all()
  Run

---

• PMunch (orginal) [2023-12-18T13:02:15+01:00] view original

  You seem to be looking for all (https://nim-lang.org/docs/asyncfutures.html#all,varargs[Future[T]]). It takes a series of futures and return a future that will resolve when all the others are done.

---

• coneonthefloor (orginal) [2023-12-18T14:18:16+01:00] view original

  @amadan & @PMunch

  That is exactly what I was looking for. Thank you.

---

• coneonthefloor (orginal) [2023-12-18T14:41:10+01:00] view original

  As a note on this. By implementing the code as per @amadan's example. The benchmark execution of parsing a large sitemap went from ~9 seconds to ~2 seconds.

---