Mirror of forum.nim-lang.org

9193 :: Is `setTimer` workable for long term timers, e.g 30 days?

• cmc (orginal) [2022-06-02T12:56:51+02:00] view original

  I'm programming a daemon that periodically expires and deletes resources as part of its normal operation. Cleanup jobs are stored persistently and restarted on program start.

  If the program is not restarted and runs for a long time, is it feasible to addTimer a cleanup job to run in e.g. 30 days or so, or will the bog down system resources too much? There might be thousands of timers set eventually.

  With thanks to the knowledgable!

---

• ingo (orginal) [2022-06-02T18:17:12+02:00] view original

  Not sure wether is is suitable for your use, I had some things that needed to run 'timed' and put them all in a heap / priority queue resulting in a single timer.

---

• greenfork (orginal) [2022-06-02T18:48:22+02:00] view original

  You could probably test it. Create a thousand timers and see how it goes. If you are doing IO, this shouldn't be a problem, only OS is the limit. If you are doing CPU intensive tasks, then it depends, generally it is better to use threads instead of event loop, which is used by addTimer. You might also want to set the ulimit -n N in the terminal so that N is greater than the number of timers because each timer will take a file descriptor.

---

• Araq (orginal) [2022-06-02T22:02:48+02:00] view original

  Sounds more like you want to setup a separate service that runs every 30 days. If you're on Linux search for "systemd", for Windows search for "setup Windows service".

---

• cmc (orginal) [2022-06-02T22:13:44+02:00] view original

  Good point, done. Works fine but needs the mentioned file descriptors.

  Dang, looks like I'm going to have to resort to chosing a resolution and polling after all, I was hoping for something that only wakes up when needed.

  I could do that with one timer that triggers another, but I have no way of canceling it in the case of adding a task that is sooner than all others.

  Thanks for the input!

---

• cmc (orginal) [2022-06-02T22:35:33+02:00] view original

  Thanks for weighing in! What I'm going for is essentially a persistent table with expiry, not unlike put/get on redis. So I'm using nim-lmdb and now bolting on the expiry. Since everything is already in-process, this has to be too.

---

• cmc (orginal) [2022-06-02T22:37:47+02:00] view original

  I think I might have found an approach. If I have a single timer that waits until it's time for the next expiry and then triggers the next delta wait, that would work. But I would need a way to get rid of the current running future. I read somewhere that futures can't generally be cancelled, but this seems to fit the bill: I will wait using withTimeout, and if the future needs to end early, I will complete it.

---

• greenfork (orginal) [2022-06-03T07:38:53+02:00] view original

  I still don't know if an async future is designed to be waited on for extended periods of time.

  They are not designed for this purpose, although it doesn't make them unusable for long periods of time. Usually you want something persistent for long periods of time. See Araq's answer. Another option is to use persistent database and do timer logic there via polling or pub/sub. Designing for a system that never reboots could be naive, usually they do reboot at least sometimes.

---

• cmc (orginal) [2022-06-03T10:54:23+02:00] view original

  Cleanup jobs are stored persistently and restarted on program start.

  As described, persistence will be done manually- so after program start task definitions are loaded from persistant storage and run or scheduled.

  They are not designed for this purpose, although it doesn't make them unusable for long periods of time.

  Great, that's exactly what I needed to know. So it's not crazy but off-label use.

  So now I know that, and that and I need to re-use one future. That was everything I needed, along with some possible alternate routes- thank you very much!

---