Mirror of forum.nim-lang.org

11505 :: Introducing nginwho - An nginx log parser, Cloudflare IP restorer, and untrusted traffic blocker

• thegraynode (orginal) [2024-04-27T16:24:40+02:00] view original

  Hi all,

  I've been admiring Nim and have written some programs in it. Recently, we had to tackle a problem at work and decided to go with Nim. That is how nginwho came to life and we are gladly opensourcing it.

  nginwho is a lightweight, efficient and extremely fast program offering three main features at its core:

  1. nginx log parser: Stores nginx logs into a sqlite3 database for further analysis and actions
  2. Restore Cloudflare original visitor IP: Continuously parses Cloudflare CIDRs (IPv4 and IPv6) through their APIs so that nginx can leverage it to restore the original IP address of visitors
  3. Block untrusted requests: Uses nftables to block HTTP and HTTPS requests coming from unknown IP addresses

  Here is the link to the repo:

  https://github.com/pouriyajamshidi/nginwho

  And the link to nimble:

  https://nimble.directory/pkg/nginwho

---

• thegraynode (orginal) [2024-11-18T17:39:15+01:00] view original

  After a few weeks of work, version 2 is out. Here is the summary of changes:

  • implement reporting using --report flag
  • add v1 to v2 database migration support (uses 1/10th of disk space comparing to the previous method)
  • major cleanup of existing code
  • remove trailing '/'s from referrer
  • remove trailing '/' from URLs
  • add a warning for inclusion of nginwho addition to nginx file
  • optimize log file handling and minor cleanups
  • collect types in one file
  • replace types and improve functions separation logic
  • implement performant DB log insertion and avoid duplicate log inserts
  • fix date parsing bug introduced in Nginx 1.24.0
  • re-design database operations logic for inserts
  • better handling on log file reads to save CPU cycles
  • compiled with Nim 2.2.0

---